mm: Fix int overflow in callers of do_shrink_slab()

Submitted by Kirill Tkhai on Sept. 28, 2018, 11:33 a.m.

Details

Message ID 153813440871.4482.881903921858059737.stgit@localhost.localdomain
State New
Series "mm: Fix int overflow in callers of do_shrink_slab()"
Headers show

Commit Message

Kirill Tkhai Sept. 28, 2018, 11:33 a.m.
do_shrink_slab() returns unsigned long value, and
the placing into int variable cuts high bytes off.
Then we compare ret and 0xfffffffe (since SHRINK_EMPTY
is converted to ret type).

Thus, big number of objects returned by do_shrink_slab()
may be interpreted as SHRINK_EMPTY, if low bytes of
their value are equal to 0xfffffffe. Fix that
by declaration ret as unsigned long in these functions.

Reported-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
---
 mm/vmscan.c |    7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

Patch hide | download patch | download mbox

diff --git a/mm/vmscan.c b/mm/vmscan.c
index b7a463b5e508..6be538ce81b6 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -435,8 +435,8 @@  static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid,
 			struct mem_cgroup *memcg, int priority)
 {
 	struct memcg_shrinker_map *map;
-	unsigned long freed = 0;
-	int ret, i;
+	unsigned long ret, freed = 0;
+	int i;
 
 	if (!memcg_kmem_enabled())
 		return 0;
@@ -535,9 +535,8 @@  static unsigned long shrink_slab(gfp_t gfp_mask, int nid,
 				 int priority,
 				 bool for_drop_caches)
 {
+	unsigned long ret, freed = 0;
 	struct shrinker *shrinker;
-	unsigned long freed = 0;
-	int ret;
 
 	if (unlikely(test_tsk_thread_flag(current, TIF_MEMDIE)))
 		return 0;

Comments

Kirill Gorkunov Sept. 28, 2018, 3:18 p.m.
On Fri, Sep 28, 2018 at 02:33:38PM +0300, Kirill Tkhai wrote:
> do_shrink_slab() returns unsigned long value, and
> the placing into int variable cuts high bytes off.
> Then we compare ret and 0xfffffffe (since SHRINK_EMPTY
> is converted to ret type).
> 
> Thus, big number of objects returned by do_shrink_slab()
> may be interpreted as SHRINK_EMPTY, if low bytes of
> their value are equal to 0xfffffffe. Fix that
> by declaration ret as unsigned long in these functions.
> 
> Reported-by: Cyrill Gorcunov <gorcunov@openvz.org>
> Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>