Page allocation failure on low memory systems

Which version of Madwifi are you using?

This looks like DMA memory is getting used up; there was a bug in which Fast Frame buffers were not getting freed on some architectures (which would happen under high traffic), which has been fixed. Maybe try HEAD?

mentor, i have seen this - but it is problematic to reproduce it - on more boxes/places/madwifi revisions ,so will try to use current snapshots but it is all on live network and all low memory systems are at nearly unaccesible places so i think twice about every kernel/madwifi update there. First, i will try to reproduce it on some testbed machine with current svn revision. If i properly remeber, latest revision i have seen it on has been something around r2150 or stuff but not sure, will try to dig more info and report

OK, well I'll dig at the error a bit more then.

bad news - it is still reproducable on svn r2279

scenario :

windoze box with StrongDC - dc client, good for repruducing this stuff - installed <-> connected by lan cable to RB532 with 2.6.19.7 kernel - 2.6.20.X is unstable there - 32MB of memory, 4 atheros minipci cards, one active <-> connected to madwifi powered AP - box with 9 atheros cards in minipci2pci adapters, 2.6.20.7 kernel, latest trunk revision and that connected <-> to 20mbit/s FD internet connectivity

important is of course the RB532 device - so, only one madwifi link is active here, and it is one cm10 on RB532 in sta mode <-> one cm9 on amd based based box described above

when man connect to some DC hubs and start search and also in meanwhile start downloading something - let us say linux isos - many connections are opened and sooner or latter - matter of seconds or few minutes - RB532 aka memory constrained stop respond from AMD box and throw following stuff into syslog. Ifconfig down/up on AMD box - so on atheros interface i master mode - cures it and it will all work again. Part of syslog file follows :

BTW, for three days, i have tried on same setup simply downloading at full speed - downloaded on that windoze machine from local ftp here, throughtput has been about 2,4MB/s and no problems like this, so i guess it is not matter of load but of opening and closing connections or something like that, just RX FIFO overrun, resseting all the time, which is weird but imo not related

Apr 24 13:09:47 zmen_mne kernel: wifi0: rx FIFO overrun; resetting
Apr 24 13:11:29 zmen_mne last message repeated 2 times
Apr 24 13:12:02 zmen_mne last message repeated 2 times
Apr 24 13:13:38 zmen_mne last message repeated 4 times
Apr 24 13:14:43 zmen_mne last message repeated 2 times
Apr 24 13:14:47 zmen_mne kernel: wifi0: rx FIFO overrun; resetting
Apr 24 13:16:42 zmen_mne last message repeated 4 times
Apr 24 13:17:21 zmen_mne last message repeated 2 times
Apr 24 13:18:46 zmen_mne kernel: wifi0: rx FIFO overrun; resetting
Apr 24 13:19:43 zmen_mne last message repeated 7 times
Apr 24 13:21:04 zmen_mne last message repeated 4 times
Apr 24 13:21:50 zmen_mne last message repeated 3 times
Apr 24 13:23:06 zmen_mne last message repeated 3 times
Apr 24 13:24:14 zmen_mne kernel: wifi0: rx FIFO overrun; resetting
Apr 24 13:25:13 zmen_mne last message repeated 4 times
Apr 24 13:26:00 zmen_mne last message repeated 3 times
Apr 24 13:26:59 zmen_mne last message repeated 4 times
Apr 24 13:28:41 zmen_mne kernel: wifi0: rx FIFO overrun; resetting
Apr 24 13:28:56 zmen_mne kernel: wifi0: rx FIFO overrun; resetting
Apr 24 13:29:57 zmen_mne last message repeated 2 times
Apr 24 13:31:29 zmen_mne last message repeated 2 times
Apr 24 13:32:34 zmen_mne last message repeated 4 times
Apr 24 13:33:49 zmen_mne kernel: wifi0: rx FIFO overrun; resetting
Apr 24 13:34:30 zmen_mne kernel: wifi0: rx FIFO overrun; resetting
Apr 24 13:36:26 zmen_mne kernel: wifi0: rx FIFO overrun; resetting
Apr 24 13:37:44 zmen_mne last message repeated 2 times
Apr 24 13:38:20 zmen_mne kernel: wifi0: rx FIFO overrun; resetting

at 13:38 i stopped transfer
at about 17:40 i started search and download

Apr 24 17:41:05 zmen_mne kernel: Call Trace:
Apr 24 17:41:05 zmen_mne kernel: [<8010956c>] dump_stack+0x8/0x34
Apr 24 17:41:05 zmen_mne kernel: [<8015e4b0>] __alloc_pages+0x2a0/0x348
Apr 24 17:41:05 zmen_mne kernel: [<8017f374>] cache_alloc_refill+0x340/0x7bc
Apr 24 17:41:05 zmen_mne kernel: [<8017f02c>] kmem_cache_alloc+0x90/0x98
Apr 24 17:41:05 zmen_mne kernel: [<c018fa9c>] ip_conntrack_alloc+0xa0/0x538 [ip_conntrack]
Apr 24 17:41:05 zmen_mne kernel: [<c01902fc>] ip_conntrack_in+0x3c8/0x888 [ip_conntrack]
Apr 24 17:41:05 zmen_mne kernel: [<803207ac>] nf_iterate+0xbc/0x130
Apr 24 17:41:05 zmen_mne kernel: [<803208bc>] nf_hook_slow+0x9c/0x1a8
Apr 24 17:41:05 zmen_mne kernel: [<8032b518>] ip_rcv+0x25c/0x7a4
Apr 24 17:41:05 zmen_mne kernel: [<802fe514>] netif_receive_skb+0x460/0x784
Apr 24 17:41:05 zmen_mne kernel: [<802fe94c>] process_backlog+0x114/0x2fc
Apr 24 17:41:05 zmen_mne kernel: [<802fec10>] net_rx_action+0xdc/0x2c0
Apr 24 17:41:05 zmen_mne kernel: [<8012eda8>] __do_softirq+0x8c/0x134
Apr 24 17:41:05 zmen_mne kernel: [<8012eedc>] do_softirq+0x8c/0xb8
Apr 24 17:41:05 zmen_mne kernel: [<80104278>] do_IRQ+0x24/0x34
Apr 24 17:41:05 zmen_mne kernel: [<80102880>] ret_from_irq+0x0/0x8
Apr 24 17:41:05 zmen_mne kernel: [<80164f98>] shrink_zone+0x198/0x1a2c
Apr 24 17:41:05 zmen_mne kernel: [<80166dc0>] kswapd+0x2d8/0x450
Apr 24 17:41:05 zmen_mne kernel: [<801435bc>] kthread+0x284/0x2cc
Apr 24 17:41:05 zmen_mne kernel: [<801049f0>] kernel_thread_helper+0x10/0x18
Apr 24 17:41:05 zmen_mne kernel: 
Apr 24 17:41:05 zmen_mne kernel: Mem-info:
Apr 24 17:41:05 zmen_mne kernel: DMA per-cpu:
Apr 24 17:41:05 zmen_mne kernel: CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
Apr 24 17:41:05 zmen_mne kernel: Active:3101 inactive:1485 dirty:0 writeback:0 unstable:0 free:68 slab:1762 mapped:1003 pagetables:42
Apr 24 17:41:05 zmen_mne kernel: DMA free:272kB min:720kB low:900kB high:1080kB active:12404kB inactive:5940kB present:32512kB pages_scanned:116 all_unreclaimable? no
Apr 24 17:41:05 zmen_mne kernel: lowmem_reserve[]: 0 0
Apr 24 17:41:05 zmen_mne kernel: DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 272kB
Apr 24 17:41:05 zmen_mne kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
Apr 24 17:41:05 zmen_mne kernel: Free swap  = 147160kB
Apr 24 17:41:05 zmen_mne kernel: Total swap = 147160kB
Apr 24 17:41:05 zmen_mne kernel: Free swap:       147160kB
Apr 24 17:41:05 zmen_mne kernel: 8191 pages of RAM
Apr 24 17:41:05 zmen_mne kernel: 0 pages of HIGHMEM
Apr 24 17:41:05 zmen_mne kernel: 1180 reserved pages
Apr 24 17:41:05 zmen_mne kernel: 4807 pages shared
Apr 24 17:41:05 zmen_mne kernel: 0 pages swap cached
Apr 24 17:41:05 zmen_mne kernel: kswapd0: page allocation failure. order:0, mode:0x20
Apr 24 17:41:05 zmen_mne kernel: Call Trace:
Apr 24 17:41:05 zmen_mne kernel: [<8010956c>] dump_stack+0x8/0x34
Apr 24 17:41:05 zmen_mne kernel: [<8015e4b0>] __alloc_pages+0x2a0/0x348
Apr 24 17:41:05 zmen_mne kernel: [<8017f374>] cache_alloc_refill+0x340/0x7bc
Apr 24 17:41:05 zmen_mne kernel: [<8017f02c>] kmem_cache_alloc+0x90/0x98
Apr 24 17:41:05 zmen_mne kernel: [<c018fa9c>] ip_conntrack_alloc+0xa0/0x538 [ip_conntrack]
Apr 24 17:41:05 zmen_mne kernel: [<c01902fc>] ip_conntrack_in+0x3c8/0x888 [ip_conntrack]
Apr 24 17:41:05 zmen_mne kernel: [<803207ac>] nf_iterate+0xbc/0x130
Apr 24 17:41:05 zmen_mne kernel: [<803208bc>] nf_hook_slow+0x9c/0x1a8
Apr 24 17:41:05 zmen_mne kernel: [<8032b518>] ip_rcv+0x25c/0x7a4
Apr 24 17:41:05 zmen_mne kernel: [<802fe514>] netif_receive_skb+0x460/0x784
Apr 24 17:41:05 zmen_mne kernel: [<802fe94c>] process_backlog+0x114/0x2fc
Apr 24 17:41:05 zmen_mne kernel: [<802fec10>] net_rx_action+0xdc/0x2c0
Apr 24 17:41:05 zmen_mne kernel: [<8012eda8>] __do_softirq+0x8c/0x134
Apr 24 17:41:05 zmen_mne kernel: [<8012eedc>] do_softirq+0x8c/0xb8
Apr 24 17:41:05 zmen_mne kernel: [<80104278>] do_IRQ+0x24/0x34
Apr 24 17:41:05 zmen_mne kernel: [<80102880>] ret_from_irq+0x0/0x8
Apr 24 17:41:05 zmen_mne kernel: [<80164f98>] shrink_zone+0x198/0x1a2c
Apr 24 17:41:05 zmen_mne kernel: [<80166dc0>] kswapd+0x2d8/0x450
Apr 24 17:41:05 zmen_mne kernel: [<801435bc>] kthread+0x284/0x2cc
Apr 24 17:41:05 zmen_mne kernel: [<801049f0>] kernel_thread_helper+0x10/0x18
Apr 24 17:41:05 zmen_mne kernel: 
Apr 24 17:41:05 zmen_mne kernel: Mem-info:
Apr 24 17:41:05 zmen_mne kernel: DMA per-cpu:
Apr 24 17:41:05 zmen_mne kernel: CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
Apr 24 17:41:05 zmen_mne kernel: Active:3101 inactive:1485 dirty:0 writeback:0 unstable:0 free:68 slab:1762 mapped:1003 pagetables:42
Apr 24 17:41:05 zmen_mne kernel: DMA free:272kB min:720kB low:900kB high:1080kB active:12404kB inactive:5940kB present:32512kB pages_scanned:116 all_unreclaimable? no
Apr 24 17:41:05 zmen_mne kernel: lowmem_reserve[]: 0 0
Apr 24 17:41:05 zmen_mne kernel: DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 272kB
Apr 24 17:41:05 zmen_mne kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
Apr 24 17:41:05 zmen_mne kernel: Free swap  = 147160kB
Apr 24 17:41:05 zmen_mne kernel: Total swap = 147160kB
Apr 24 17:41:05 zmen_mne kernel: Free swap:       147160kB
Apr 24 17:41:05 zmen_mne kernel: 8191 pages of RAM
Apr 24 17:41:05 zmen_mne kernel: 0 pages of HIGHMEM
Apr 24 17:41:05 zmen_mne kernel: 1180 reserved pages
Apr 24 17:41:05 zmen_mne kernel: 4807 pages shared
Apr 24 17:41:05 zmen_mne kernel: 0 pages swap cached
Apr 24 17:41:05 zmen_mne kernel: kswapd0: page allocation failure. order:0, mode:0x20
Apr 24 17:41:05 zmen_mne kernel: Call Trace:
Apr 24 17:41:05 zmen_mne kernel: [<8010956c>] dump_stack+0x8/0x34
Apr 24 17:41:05 zmen_mne kernel: [<8015e4b0>] __alloc_pages+0x2a0/0x348
Apr 24 17:41:05 zmen_mne kernel: [<8017f374>] cache_alloc_refill+0x340/0x7bc
Apr 24 17:41:05 zmen_mne kernel: [<8017f02c>] kmem_cache_alloc+0x90/0x98
Apr 24 17:41:05 zmen_mne kernel: [<c018fa9c>] ip_conntrack_alloc+0xa0/0x538 [ip_conntrack]
Apr 24 17:41:05 zmen_mne kernel: [<c01902fc>] ip_conntrack_in+0x3c8/0x888 [ip_conntrack]
Apr 24 17:41:05 zmen_mne kernel: [<803207ac>] nf_iterate+0xbc/0x130
Apr 24 17:41:05 zmen_mne kernel: [<803208bc>] nf_hook_slow+0x9c/0x1a8
Apr 24 17:41:05 zmen_mne kernel: [<8032b518>] ip_rcv+0x25c/0x7a4
Apr 24 17:41:05 zmen_mne kernel: [<802fe514>] netif_receive_skb+0x460/0x784
Apr 24 17:41:05 zmen_mne kernel: [<802fe94c>] process_backlog+0x114/0x2fc
Apr 24 17:41:05 zmen_mne kernel: [<802fec10>] net_rx_action+0xdc/0x2c0
Apr 24 17:41:05 zmen_mne kernel: [<8012eda8>] __do_softirq+0x8c/0x134
Apr 24 17:41:05 zmen_mne kernel: [<8012eedc>] do_softirq+0x8c/0xb8
Apr 24 17:41:05 zmen_mne kernel: [<80104278>] do_IRQ+0x24/0x34
Apr 24 17:41:05 zmen_mne kernel: [<80102880>] ret_from_irq+0x0/0x8
Apr 24 17:41:05 zmen_mne kernel: [<80164f98>] shrink_zone+0x198/0x1a2c
Apr 24 17:41:05 zmen_mne kernel: [<80166dc0>] kswapd+0x2d8/0x450
Apr 24 17:41:05 zmen_mne kernel: [<801435bc>] kthread+0x284/0x2cc
Apr 24 17:41:05 zmen_mne kernel: [<801049f0>] kernel_thread_helper+0x10/0x18
Apr 24 17:41:05 zmen_mne kernel: 
Apr 24 17:41:05 zmen_mne kernel: Mem-info:
Apr 24 17:41:05 zmen_mne kernel: DMA per-cpu:
Apr 24 17:41:05 zmen_mne kernel: CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
Apr 24 17:41:05 zmen_mne kernel: Active:3101 inactive:1485 dirty:0 writeback:0 unstable:0 free:68 slab:1762 mapped:1003 pagetables:42
Apr 24 17:41:05 zmen_mne kernel: DMA free:272kB min:720kB low:900kB high:1080kB active:12404kB inactive:5940kB present:32512kB pages_scanned:116 all_unreclaimable? no
Apr 24 17:41:05 zmen_mne kernel: lowmem_reserve[]: 0 0
Apr 24 17:41:05 zmen_mne kernel: DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 272kB
Apr 24 17:41:05 zmen_mne kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
Apr 24 17:41:05 zmen_mne kernel: Free swap  = 147160kB
Apr 24 17:41:05 zmen_mne kernel: Total swap = 147160kB
Apr 24 17:41:05 zmen_mne kernel: Free swap:       147160kB
Apr 24 17:41:05 zmen_mne kernel: 8191 pages of RAM
Apr 24 17:41:05 zmen_mne kernel: 0 pages of HIGHMEM
Apr 24 17:41:05 zmen_mne kernel: 1180 reserved pages
Apr 24 17:41:05 zmen_mne kernel: 4807 pages shared
Apr 24 17:41:05 zmen_mne kernel: 0 pages swap cached
Apr 24 17:41:05 zmen_mne kernel: kswapd0: page allocation failure. order:0, mode:0x20
Apr 24 17:41:05 zmen_mne kernel: Call Trace:
Apr 24 17:41:05 zmen_mne kernel: [<8010956c>] dump_stack+0x8/0x34
Apr 24 17:41:05 zmen_mne kernel: [<8015e4b0>] __alloc_pages+0x2a0/0x348
Apr 24 17:41:05 zmen_mne kernel: [<8017f374>] cache_alloc_refill+0x340/0x7bc
Apr 24 17:41:05 zmen_mne kernel: [<8017f02c>] kmem_cache_alloc+0x90/0x98
Apr 24 17:41:05 zmen_mne kernel: [<c018fa9c>] ip_conntrack_alloc+0xa0/0x538 [ip_conntrack]
Apr 24 17:41:05 zmen_mne kernel: [<c01902fc>] ip_conntrack_in+0x3c8/0x888 [ip_conntrack]
Apr 24 17:41:05 zmen_mne kernel: [<803207ac>] nf_iterate+0xbc/0x130
Apr 24 17:41:05 zmen_mne kernel: [<803208bc>] nf_hook_slow+0x9c/0x1a8
Apr 24 17:41:05 zmen_mne kernel: [<8032b518>] ip_rcv+0x25c/0x7a4
Apr 24 17:41:05 zmen_mne kernel: [<802fe514>] netif_receive_skb+0x460/0x784
Apr 24 17:41:05 zmen_mne kernel: [<802fe94c>] process_backlog+0x114/0x2fc
Apr 24 17:41:05 zmen_mne kernel: [<802fec10>] net_rx_action+0xdc/0x2c0
Apr 24 17:41:05 zmen_mne kernel: [<8012eda8>] __do_softirq+0x8c/0x134
Apr 24 17:41:05 zmen_mne kernel: [<8012eedc>] do_softirq+0x8c/0xb8
Apr 24 17:41:05 zmen_mne kernel: [<80104278>] do_IRQ+0x24/0x34
Apr 24 17:41:05 zmen_mne kernel: [<80102880>] ret_from_irq+0x0/0x8
Apr 24 17:41:05 zmen_mne kernel: [<80164f98>] shrink_zone+0x198/0x1a2c
Apr 24 17:41:05 zmen_mne kernel: [<80166dc0>] kswapd+0x2d8/0x450
Apr 24 17:41:05 zmen_mne kernel: [<801435bc>] kthread+0x284/0x2cc
Apr 24 17:41:05 zmen_mne kernel: [<801049f0>] kernel_thread_helper+0x10/0x18
Apr 24 17:41:05 zmen_mne kernel: 
Apr 24 17:41:05 zmen_mne kernel: Mem-info:
Apr 24 17:41:05 zmen_mne kernel: DMA per-cpu:
Apr 24 17:41:05 zmen_mne kernel: CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
Apr 24 17:41:05 zmen_mne kernel: Active:3101 inactive:1485 dirty:0 writeback:0 unstable:0 free:68 slab:1762 mapped:1003 pagetables:42
Apr 24 17:41:05 zmen_mne kernel: DMA free:272kB min:720kB low:900kB high:1080kB active:12404kB inactive:5940kB present:32512kB pages_scanned:116 all_unreclaimable? no
Apr 24 17:41:05 zmen_mne kernel: lowmem_reserve[]: 0 0
Apr 24 17:41:05 zmen_mne kernel: DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 272kB
Apr 24 17:41:05 zmen_mne kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
Apr 24 17:41:05 zmen_mne kernel: Free swap  = 147160kB
Apr 24 17:41:05 zmen_mne kernel: Total swap = 147160kB
Apr 24 17:41:05 zmen_mne kernel: Free swap:       147160kB
Apr 24 17:41:05 zmen_mne kernel: 8191 pages of RAM
Apr 24 17:41:05 zmen_mne kernel: 0 pages of HIGHMEM
Apr 24 17:41:05 zmen_mne kernel: 1180 reserved pages
Apr 24 17:41:05 zmen_mne kernel: 4807 pages shared
Apr 24 17:41:05 zmen_mne kernel: 0 pages swap cached
Apr 24 17:41:05 zmen_mne kernel: kswapd0: page allocation failure. order:0, mode:0x20
Apr 24 17:41:05 zmen_mne kernel: Call Trace:
Apr 24 17:41:05 zmen_mne kernel: [<8010956c>] dump_stack+0x8/0x34
Apr 24 17:41:05 zmen_mne kernel: [<8015e4b0>] __alloc_pages+0x2a0/0x348
Apr 24 17:41:05 zmen_mne kernel: [<8017f374>] cache_alloc_refill+0x340/0x7bc
Apr 24 17:41:05 zmen_mne kernel: [<8017f02c>] kmem_cache_alloc+0x90/0x98
Apr 24 17:41:05 zmen_mne kernel: [<c018fa9c>] ip_conntrack_alloc+0xa0/0x538 [ip_conntrack]
Apr 24 17:41:05 zmen_mne kernel: [<c01902fc>] ip_conntrack_in+0x3c8/0x888 [ip_conntrack]
Apr 24 17:41:05 zmen_mne kernel: [<803207ac>] nf_iterate+0xbc/0x130
Apr 24 17:41:05 zmen_mne kernel: [<803208bc>] nf_hook_slow+0x9c/0x1a8
Apr 24 17:41:05 zmen_mne kernel: [<8032b518>] ip_rcv+0x25c/0x7a4
Apr 24 17:41:05 zmen_mne kernel: [<802fe514>] netif_receive_skb+0x460/0x784
Apr 24 17:41:05 zmen_mne kernel: [<802fe94c>] process_backlog+0x114/0x2fc
Apr 24 17:41:05 zmen_mne kernel: [<802fec10>] net_rx_action+0xdc/0x2c0
Apr 24 17:41:05 zmen_mne kernel: [<8012eda8>] __do_softirq+0x8c/0x134
Apr 24 17:41:05 zmen_mne kernel: [<8012eedc>] do_softirq+0x8c/0xb8
Apr 24 17:41:05 zmen_mne kernel: [<80104278>] do_IRQ+0x24/0x34
Apr 24 17:41:05 zmen_mne kernel: [<80102880>] ret_from_irq+0x0/0x8
Apr 24 17:41:05 zmen_mne kernel: [<80164f98>] shrink_zone+0x198/0x1a2c
Apr 24 17:41:05 zmen_mne kernel: [<80166dc0>] kswapd+0x2d8/0x450
Apr 24 17:41:05 zmen_mne kernel: [<801435bc>] kthread+0x284/0x2cc
Apr 24 17:41:05 zmen_mne kernel: [<801049f0>] kernel_thread_helper+0x10/0x18
Apr 24 17:41:05 zmen_mne kernel: 
Apr 24 17:41:05 zmen_mne kernel: Mem-info:
Apr 24 17:41:05 zmen_mne kernel: DMA per-cpu:
Apr 24 17:41:05 zmen_mne kernel: CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
Apr 24 17:41:05 zmen_mne kernel: Active:3101 inactive:1485 dirty:0 writeback:0 unstable:0 free:68 slab:1762 mapped:1003 pagetables:42
Apr 24 17:41:05 zmen_mne kernel: DMA free:272kB min:720kB low:900kB high:1080kB active:12404kB inactive:5940kB present:32512kB pages_scanned:116 all_unreclaimable? no
Apr 24 17:41:05 zmen_mne kernel: lowmem_reserve[]: 0 0
Apr 24 17:41:05 zmen_mne kernel: DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 272kB
Apr 24 17:41:05 zmen_mne kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
Apr 24 17:41:05 zmen_mne kernel: Free swap  = 147160kB
Apr 24 17:41:05 zmen_mne kernel: Total swap = 147160kB
Apr 24 17:41:05 zmen_mne kernel: Free swap:       147160kB
Apr 24 17:41:05 zmen_mne kernel: 8191 pages of RAM
Apr 24 17:41:05 zmen_mne kernel: 0 pages of HIGHMEM
Apr 24 17:41:05 zmen_mne kernel: 1180 reserved pages
Apr 24 17:41:05 zmen_mne kernel: 4807 pages shared
Apr 24 17:41:05 zmen_mne kernel: 0 pages swap cached
Apr 24 17:41:05 zmen_mne kernel: kswapd0: page allocation failure. order:0, mode:0x20
Apr 24 17:41:05 zmen_mne kernel: Call Trace:
Apr 24 17:41:06 zmen_mne kernel: [<8010956c>] dump_stack+0x8/0x34
Apr 24 17:41:06 zmen_mne kernel: [<8015e4b0>] __alloc_pages+0x2a0/0x348
Apr 24 17:41:06 zmen_mne kernel: [<8017f374>] cache_alloc_refill+0x340/0x7bc
Apr 24 17:41:06 zmen_mne kernel: [<8017f02c>] kmem_cache_alloc+0x90/0x98
Apr 24 17:41:06 zmen_mne kernel: [<c018fa9c>] ip_conntrack_alloc+0xa0/0x538 [ip_conntrack]
Apr 24 17:41:06 zmen_mne kernel: [<c01902fc>] ip_conntrack_in+0x3c8/0x888 [ip_conntrack]
Apr 24 17:41:06 zmen_mne kernel: [<803207ac>] nf_iterate+0xbc/0x130
Apr 24 17:41:06 zmen_mne kernel: [<803208bc>] nf_hook_slow+0x9c/0x1a8
Apr 24 17:41:06 zmen_mne kernel: [<8032b518>] ip_rcv+0x25c/0x7a4
Apr 24 17:41:06 zmen_mne kernel: [<802fe514>] netif_receive_skb+0x460/0x784
Apr 24 17:41:06 zmen_mne kernel: [<802fe94c>] process_backlog+0x114/0x2fc
Apr 24 17:41:06 zmen_mne kernel: [<802fec10>] net_rx_action+0xdc/0x2c0
Apr 24 17:41:06 zmen_mne kernel: [<8012eda8>] __do_softirq+0x8c/0x134
Apr 24 17:41:06 zmen_mne kernel: [<8012eedc>] do_softirq+0x8c/0xb8
Apr 24 17:41:06 zmen_mne kernel: [<80104278>] do_IRQ+0x24/0x34
Apr 24 17:41:06 zmen_mne kernel: [<80102880>] ret_from_irq+0x0/0x8
Apr 24 17:41:06 zmen_mne kernel: [<80164f98>] shrink_zone+0x198/0x1a2c
Apr 24 17:41:06 zmen_mne kernel: [<80166dc0>] kswapd+0x2d8/0x450
Apr 24 17:41:06 zmen_mne kernel: [<801435bc>] kthread+0x284/0x2cc
Apr 24 17:41:06 zmen_mne kernel: [<801049f0>] kernel_thread_helper+0x10/0x18
Apr 24 17:41:06 zmen_mne kernel: 
Apr 24 17:41:06 zmen_mne kernel: Mem-info:
Apr 24 17:41:06 zmen_mne kernel: DMA per-cpu:
Apr 24 17:41:06 zmen_mne kernel: CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
Apr 24 17:41:06 zmen_mne kernel: Active:3101 inactive:1485 dirty:0 writeback:0 unstable:0 free:68 slab:1762 mapped:1003 pagetables:42
Apr 24 17:41:06 zmen_mne kernel: DMA free:272kB min:720kB low:900kB high:1080kB active:12404kB inactive:5940kB present:32512kB pages_scanned:116 all_unreclaimable? no
Apr 24 17:41:06 zmen_mne kernel: lowmem_reserve[]: 0 0
Apr 24 17:41:06 zmen_mne kernel: DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 272kB
Apr 24 17:41:06 zmen_mne kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
Apr 24 17:41:06 zmen_mne kernel: Free swap  = 147160kB
Apr 24 17:41:06 zmen_mne kernel: Total swap = 147160kB
Apr 24 17:41:06 zmen_mne kernel: Free swap:       147160kB
Apr 24 17:41:06 zmen_mne kernel: 8191 pages of RAM
Apr 24 17:41:06 zmen_mne kernel: 0 pages of HIGHMEM
Apr 24 17:41:06 zmen_mne kernel: 1180 reserved pages
Apr 24 17:41:06 zmen_mne kernel: 4807 pages shared
Apr 24 17:41:06 zmen_mne kernel: 0 pages swap cached
Apr 24 17:41:06 zmen_mne kernel: events/0: page allocation failure. order:0, mode:0x20
Apr 24 17:41:06 zmen_mne kernel: Call Trace:
Apr 24 17:41:06 zmen_mne kernel: [<8010956c>] dump_stack+0x8/0x34
Apr 24 17:41:06 zmen_mne kernel: [<8015e4b0>] __alloc_pages+0x2a0/0x348
Apr 24 17:41:06 zmen_mne kernel: [<8017f374>] cache_alloc_refill+0x340/0x7bc
Apr 24 17:41:06 zmen_mne kernel: [<8017f02c>] kmem_cache_alloc+0x90/0x98
Apr 24 17:41:06 zmen_mne kernel: [<c018fa9c>] ip_conntrack_alloc+0xa0/0x538 [ip_conntrack]
Apr 24 17:41:06 zmen_mne kernel: [<c01902fc>] ip_conntrack_in+0x3c8/0x888 [ip_conntrack]
Apr 24 17:41:06 zmen_mne kernel: [<803207ac>] nf_iterate+0xbc/0x130
Apr 24 17:41:06 zmen_mne kernel: [<803208bc>] nf_hook_slow+0x9c/0x1a8
Apr 24 17:41:06 zmen_mne kernel: [<8032b518>] ip_rcv+0x25c/0x7a4
Apr 24 17:41:06 zmen_mne kernel: [<802fe514>] netif_receive_skb+0x460/0x784
Apr 24 17:41:06 zmen_mne kernel: [<802fe94c>] process_backlog+0x114/0x2fc
Apr 24 17:41:06 zmen_mne kernel: [<802fec10>] net_rx_action+0xdc/0x2c0
Apr 24 17:41:06 zmen_mne kernel: [<8012eda8>] __do_softirq+0x8c/0x134
Apr 24 17:41:06 zmen_mne kernel: [<8012eedc>] do_softirq+0x8c/0xb8
Apr 24 17:41:06 zmen_mne kernel: [<80104278>] do_IRQ+0x24/0x34
Apr 24 17:41:06 zmen_mne kernel: [<80102880>] ret_from_irq+0x0/0x8
Apr 24 17:41:06 zmen_mne kernel: [<8013e548>] run_workqueue+0x204/0x264
Apr 24 17:41:06 zmen_mne kernel: [<8013e710>] worker_thread+0x168/0x198
Apr 24 17:41:06 zmen_mne kernel: [<801435bc>] kthread+0x284/0x2cc
Apr 24 17:41:06 zmen_mne kernel: [<801049f0>] kernel_thread_helper+0x10/0x18
Apr 24 17:41:06 zmen_mne kernel: 
Apr 24 17:41:06 zmen_mne kernel: Mem-info:
Apr 24 17:41:06 zmen_mne kernel: DMA per-cpu:
Apr 24 17:41:06 zmen_mne kernel: CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
Apr 24 17:41:06 zmen_mne kernel: Active:3101 inactive:1485 dirty:0 writeback:0 unstable:0 free:68 slab:1762 mapped:1003 pagetables:42
Apr 24 17:41:06 zmen_mne kernel: DMA free:272kB min:720kB low:900kB high:1080kB active:12404kB inactive:5940kB present:32512kB pages_scanned:116 all_unreclaimable? no
Apr 24 17:41:06 zmen_mne kernel: lowmem_reserve[]: 0 0
Apr 24 17:41:06 zmen_mne kernel: DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 272kB
Apr 24 17:41:06 zmen_mne kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
Apr 24 17:41:06 zmen_mne kernel: Free swap  = 147160kB
Apr 24 17:41:06 zmen_mne kernel: Total swap = 147160kB
Apr 24 17:41:06 zmen_mne kernel: Free swap:       147160kB
Apr 24 17:41:06 zmen_mne kernel: 8191 pages of RAM
Apr 24 17:41:06 zmen_mne kernel: 0 pages of HIGHMEM
Apr 24 17:41:06 zmen_mne kernel: 1180 reserved pages
Apr 24 17:41:06 zmen_mne kernel: 4807 pages shared
Apr 24 17:41:06 zmen_mne kernel: 0 pages swap cached
Apr 24 17:41:06 zmen_mne kernel: events/0: page allocation failure. order:0, mode:0x20
Apr 24 17:41:06 zmen_mne kernel: Call Trace:
Apr 24 17:41:06 zmen_mne kernel: [<8010956c>] dump_stack+0x8/0x34
Apr 24 17:41:06 zmen_mne kernel: [<8015e4b0>] __alloc_pages+0x2a0/0x348
Apr 24 17:41:06 zmen_mne kernel: [<8017f374>] cache_alloc_refill+0x340/0x7bc
Apr 24 17:41:06 zmen_mne kernel: [<8017f02c>] kmem_cache_alloc+0x90/0x98
Apr 24 17:41:06 zmen_mne kernel: [<c018fa9c>] ip_conntrack_alloc+0xa0/0x538 [ip_conntrack]
Apr 24 17:41:06 zmen_mne kernel: [<c01902fc>] ip_conntrack_in+0x3c8/0x888 [ip_conntrack]
Apr 24 17:41:06 zmen_mne kernel: [<803207ac>] nf_iterate+0xbc/0x130
Apr 24 17:41:06 zmen_mne kernel: [<803208bc>] nf_hook_slow+0x9c/0x1a8
Apr 24 17:41:06 zmen_mne kernel: [<8032b518>] ip_rcv+0x25c/0x7a4
Apr 24 17:41:06 zmen_mne kernel: [<802fe514>] netif_receive_skb+0x460/0x784
Apr 24 17:41:06 zmen_mne kernel: [<802fe94c>] process_backlog+0x114/0x2fc
Apr 24 17:41:06 zmen_mne kernel: [<802fec10>] net_rx_action+0xdc/0x2c0
Apr 24 17:41:06 zmen_mne kernel: [<8012eda8>] __do_softirq+0x8c/0x134
Apr 24 17:41:06 zmen_mne kernel: [<8012eedc>] do_softirq+0x8c/0xb8
Apr 24 17:41:06 zmen_mne kernel: [<80104278>] do_IRQ+0x24/0x34
Apr 24 17:41:06 zmen_mne kernel: [<80102880>] ret_from_irq+0x0/0x8
Apr 24 17:41:06 zmen_mne kernel: [<8013e548>] run_workqueue+0x204/0x264
Apr 24 17:41:06 zmen_mne kernel: [<8013e710>] worker_thread+0x168/0x198
Apr 24 17:41:06 zmen_mne kernel: [<801435bc>] kthread+0x284/0x2cc
Apr 24 17:41:06 zmen_mne kernel: [<801049f0>] kernel_thread_helper+0x10/0x18
Apr 24 17:41:06 zmen_mne kernel: 
Apr 24 17:41:06 zmen_mne kernel: Mem-info:
Apr 24 17:41:06 zmen_mne kernel: DMA per-cpu:
Apr 24 17:41:06 zmen_mne kernel: CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
Apr 24 17:41:06 zmen_mne kernel: Active:3101 inactive:1485 dirty:0 writeback:0 unstable:0 free:68 slab:1762 mapped:1003 pagetables:42
Apr 24 17:41:06 zmen_mne kernel: DMA free:272kB min:720kB low:900kB high:1080kB active:12404kB inactive:5940kB present:32512kB pages_scanned:116 all_unreclaimable? no
Apr 24 17:41:06 zmen_mne kernel: lowmem_reserve[]: 0 0
Apr 24 17:41:06 zmen_mne kernel: DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 272kB
Apr 24 17:41:06 zmen_mne kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
Apr 24 17:41:06 zmen_mne kernel: Free swap  = 147160kB
Apr 24 17:41:06 zmen_mne kernel: Total swap = 147160kB
Apr 24 17:41:06 zmen_mne kernel: Free swap:       147160kB
Apr 24 17:41:06 zmen_mne kernel: 8191 pages of RAM
Apr 24 17:41:06 zmen_mne kernel: 0 pages of HIGHMEM
Apr 24 17:41:06 zmen_mne kernel: 1180 reserved pages
Apr 24 17:41:06 zmen_mne kernel: 4807 pages shared
Apr 24 17:41:06 zmen_mne kernel: 0 pages swap cached
Apr 24 17:41:06 zmen_mne kernel: events/0: page allocation failure. order:0, mode:0x20
Apr 24 17:41:06 zmen_mne kernel: Call Trace:
Apr 24 17:41:06 zmen_mne kernel: [<8010956c>] dump_stack+0x8/0x34
Apr 24 17:41:06 zmen_mne kernel: [<8015e4b0>] __alloc_pages+0x2a0/0x348
Apr 24 17:41:06 zmen_mne kernel: [<8017f374>] cache_alloc_refill+0x340/0x7bc
Apr 24 17:41:06 zmen_mne kernel: [<8017f02c>] kmem_cache_alloc+0x90/0x98
Apr 24 17:41:06 zmen_mne kernel: [<c018fa9c>] ip_conntrack_alloc+0xa0/0x538 [ip_conntrack]
Apr 24 17:41:06 zmen_mne kernel: [<c01902fc>] ip_conntrack_in+0x3c8/0x888 [ip_conntrack]
Apr 24 17:41:06 zmen_mne kernel: [<803207ac>] nf_iterate+0xbc/0x130
Apr 24 17:41:06 zmen_mne kernel: [<803208bc>] nf_hook_slow+0x9c/0x1a8
Apr 24 17:41:06 zmen_mne kernel: [<8032b518>] ip_rcv+0x25c/0x7a4
Apr 24 17:41:06 zmen_mne kernel: [<802fe514>] netif_receive_skb+0x460/0x784
Apr 24 17:41:06 zmen_mne kernel: [<802fe94c>] process_backlog+0x114/0x2fc
Apr 24 17:41:06 zmen_mne kernel: [<802fec10>] net_rx_action+0xdc/0x2c0
Apr 24 17:41:06 zmen_mne kernel: [<8012eda8>] __do_softirq+0x8c/0x134
Apr 24 17:41:06 zmen_mne kernel: [<8012eedc>] do_softirq+0x8c/0xb8
Apr 24 17:41:06 zmen_mne kernel: [<80104278>] do_IRQ+0x24/0x34
Apr 24 17:41:06 zmen_mne kernel: [<80102880>] ret_from_irq+0x0/0x8
Apr 24 17:41:06 zmen_mne kernel: [<8013e548>] run_workqueue+0x204/0x264
Apr 24 17:41:06 zmen_mne kernel: [<8013e710>] worker_thread+0x168/0x198
Apr 24 17:41:06 zmen_mne kernel: [<801435bc>] kthread+0x284/0x2cc
Apr 24 17:41:06 zmen_mne kernel: [<801049f0>] kernel_thread_helper+0x10/0x18
Apr 24 17:41:06 zmen_mne kernel: 
Apr 24 17:41:06 zmen_mne kernel: Mem-info:
Apr 24 17:41:06 zmen_mne kernel: DMA per-cpu:
Apr 24 17:41:06 zmen_mne kernel: CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
Apr 24 17:41:06 zmen_mne kernel: Active:3101 inactive:1485 dirty:0 writeback:0 unstable:0 free:68 slab:1762 mapped:1003 pagetables:42
Apr 24 17:41:06 zmen_mne kernel: DMA free:272kB min:720kB low:900kB high:1080kB active:12404kB inactive:5940kB present:32512kB pages_scanned:116 all_unreclaimable? no
Apr 24 17:41:06 zmen_mne kernel: lowmem_reserve[]: 0 0
Apr 24 17:41:06 zmen_mne kernel: DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 272kB
Apr 24 17:41:06 zmen_mne kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
Apr 24 17:41:06 zmen_mne kernel: Free swap  = 147160kB
Apr 24 17:41:06 zmen_mne kernel: Total swap = 147160kB
Apr 24 17:41:06 zmen_mne kernel: Free swap:       147160kB
Apr 24 17:41:06 zmen_mne kernel: 8191 pages of RAM
Apr 24 17:41:06 zmen_mne kernel: 0 pages of HIGHMEM
Apr 24 17:41:06 zmen_mne kernel: 1180 reserved pages
Apr 24 17:41:06 zmen_mne kernel: 4807 pages shared
Apr 24 17:41:06 zmen_mne kernel: 0 pages swap cached
Apr 24 17:41:06 zmen_mne kernel: events/0: page allocation failure. order:0, mode:0x20
Apr 24 17:41:06 zmen_mne kernel: Call Trace:
Apr 24 17:41:06 zmen_mne kernel: [<8010956c>] dump_stack+0x8/0x34
Apr 24 17:41:06 zmen_mne kernel: [<8015e4b0>] __alloc_pages+0x2a0/0x348
Apr 24 17:41:06 zmen_mne kernel: [<8017f374>] cache_alloc_refill+0x340/0x7bc
Apr 24 17:41:06 zmen_mne kernel: [<8017f02c>] kmem_cache_alloc+0x90/0x98
Apr 24 17:41:06 zmen_mne kernel: [<c018fa9c>] ip_conntrack_alloc+0xa0/0x538 [ip_conntrack]
Apr 24 17:41:06 zmen_mne kernel: [<c01902fc>] ip_conntrack_in+0x3c8/0x888 [ip_conntrack]
Apr 24 17:41:06 zmen_mne kernel: [<803207ac>] nf_iterate+0xbc/0x130
Apr 24 17:41:06 zmen_mne kernel: [<803208bc>] nf_hook_slow+0x9c/0x1a8
Apr 24 17:41:06 zmen_mne kernel: [<8032b518>] ip_rcv+0x25c/0x7a4
Apr 24 17:41:06 zmen_mne kernel: [<802fe514>] netif_receive_skb+0x460/0x784
Apr 24 17:41:06 zmen_mne kernel: [<802fe94c>] process_backlog+0x114/0x2fc
Apr 24 17:41:06 zmen_mne kernel: [<802fec10>] net_rx_action+0xdc/0x2c0
Apr 24 17:41:06 zmen_mne kernel: [<8012eda8>] __do_softirq+0x8c/0x134
Apr 24 17:41:06 zmen_mne kernel: [<8012eedc>] do_softirq+0x8c/0xb8
Apr 24 17:41:06 zmen_mne kernel: [<80104278>] do_IRQ+0x24/0x34
Apr 24 17:41:06 zmen_mne kernel: [<80102880>] ret_from_irq+0x0/0x8
Apr 24 17:41:06 zmen_mne kernel: [<8013e548>] run_workqueue+0x204/0x264
Apr 24 17:41:06 zmen_mne kernel: [<8013e710>] worker_thread+0x168/0x198
Apr 24 17:41:06 zmen_mne kernel: [<801435bc>] kthread+0x284/0x2cc
Apr 24 17:41:06 zmen_mne kernel: [<801049f0>] kernel_thread_helper+0x10/0x18
Apr 24 17:41:06 zmen_mne kernel: 
Apr 24 17:41:06 zmen_mne kernel: Mem-info:
Apr 24 17:41:06 zmen_mne kernel: DMA per-cpu:
Apr 24 17:41:06 zmen_mne kernel: CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
Apr 24 17:41:06 zmen_mne kernel: Active:3101 inactive:1485 dirty:0 writeback:0 unstable:0 free:68 slab:1762 mapped:1003 pagetables:42
Apr 24 17:41:06 zmen_mne kernel: DMA free:272kB min:720kB low:900kB high:1080kB active:12404kB inactive:5940kB present:32512kB pages_scanned:116 all_unreclaimable? no
Apr 24 17:41:06 zmen_mne kernel: lowmem_reserve[]: 0 0
Apr 24 17:41:06 zmen_mne kernel: DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 272kB
Apr 24 17:41:06 zmen_mne kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
Apr 24 17:41:06 zmen_mne kernel: Free swap  = 147160kB
Apr 24 17:41:06 zmen_mne kernel: Total swap = 147160kB
Apr 24 17:41:06 zmen_mne kernel: Free swap:       147160kB
Apr 24 17:41:06 zmen_mne kernel: 8191 pages of RAM
Apr 24 17:41:06 zmen_mne kernel: 0 pages of HIGHMEM
Apr 24 17:41:06 zmen_mne kernel: 1180 reserved pages
Apr 24 17:41:06 zmen_mne kernel: 4807 pages shared
Apr 24 17:41:06 zmen_mne kernel: 0 pages swap cached
Apr 24 17:41:06 zmen_mne kernel: ath_uapsd_processtriggers: no skbuff

one more thing - it seems, that this only happens if both STA and AP sides of link are Madwifi, i have tried it with some HW box as AP - Trendnet TEW-510APB - to be sure and bug hasn't been triggered - but, on the other side, i only tried search in that case and not simultaneous downloading so i will try that also later, or that difference could be that before i tried it with non-public IP on that windows machine which has been NATed on RB532 and now it is pure routed public IP but it could hardly matter, anyway, there IS something weird there in driver i am afraid

ignuss

after reading kernel mailing list , it seems this is NOT bug in madwifi like memory leak or something like that, allocation failures like this could happen, problem is only that madwifi driver doesn't recover itself from it. Man has to down/up master mode side of connection, i also tried down/up on sta side of connection but that doesn't help, it is even more weird, when i do it, IP settings - we use static IPs only - somehow disappears, and anyway, we need automatic recovery am i right? So , maybe possible way of recovery could be that we put forced reassociation or even reset when "ath_uapsd_processtriggers: no skbuff" happens ? What do ya think ppl ?

BTW Mentor, i have tried to turn FF off, as ya give me hint yesterday that it could be connected with this problem, and it seems failure could be triggered even more frequently and more easy when FF are turned off.

These are all GFP_ATOMIC allocation failures that are ocurring in parts of the kernel other than Madwifi. On a second glance, they do not appear to be DMA memory leaks; that guess was complete muppetry.

This means that either Madwifi is leaking memory, or using too much memory. I know the node reference counting is horribly broken, so this may be the cause. Maybe try the madwifi-ng-refcount branch - however, it is currently broken beyond the state that trunk is.

I get a similar oops on 133 MHz Soekris board with 64MB RAM. It happens pretty reliably when the board is in monitor mode and is exposed to high-rate traffic.

1512 pages slab

45 pages pagetables swapper: page allocation failure. order:0, mode:0x20

[<c012c8bc>] alloc_pages+0x26f/0x280 [<c013a517>] cache_alloc_refill+0x214/0x3d0 [<c013a716>] kmalloc+0x43/0x4b [<c0209a44>] alloc_skb+0x51/0xfd [<c020a567>] skb_copy+0x2e/0xc3 [<c4884c5e>] ieee80211_input_monitor+0x132/0x6c1 [wlan] [<c0209a44>] alloc_skb+0x51/0xfd [<c482a41e>] ath_rx_tasklet+0x33c/0x6c3 [ath_pci] [<c0112b38>] tasklet_action+0x32/0x52 [<c0112aa0>] do_softirq+0x35/0x75 [<c0112b02>] do_softirq+0x22/0x26 [<c0103e71>] do_IRQ+0x59/0x70 [<c0257ad0>] sched_text_start+0x490/0x4f3 [<c01025b3>] common_interrupt+0x23/0x30 [<c0101329>] default_idle+0x0/0x39 [<c0101350>] default_idle+0x27/0x39 [<c0100b66>] cpu_idle+0x38/0x4d [<c02e4a83>] start_kernel+0x2d8/0x2e0 [<c02e4443>] unknown_bootoption+0x0/0x205 =======================

Mem-info: DMA per-cpu: CPU 0: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0 Normal per-cpu: CPU 0: Hot: hi: 6, btch: 1 usd: 0 Cold: hi: 2, btch: 1 usd: 1 Active:2425 inactive:11337 dirty:312 writeback:0 unstable:0

free:143 slab:1512 mapped:628 pagetables:45 bounce:0

DMA free:284kB min:252kB low:312kB high:376kB active:4kB inactive:12600kB present:16256kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 47 Normal free:288kB min:760kB low:948kB high:1140kB active:9696kB inactive:32748kB present:48768kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 DMA: 1*4kB 1*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 284kB Normal: 0*4kB 0*8kB 0*16kB 1*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 288kB Free swap: 0kB 16384 pages of RAM 0 pages of HIGHMEM 778 reserved pages 13045 pages shared 0 pages swap cached 312 pages dirty 0 pages writeback 628 pages mapped 1512 pages slab

after another testing i could say, that this behavior is still present in trunk - well, i have found another bugs in trunk during torturing this and other box which i will report later. Due to these bugs, i have tested thoroughly only trunk revision 2380 - before massive locking changes, all later trunk versions i've tried behave weirdly on memory restricted platforms i have. I have found that applying nbd's NAPI patch - from openwrt - helps a lot, but not eliminate the problem. Without that patch, bug is 100% reproducible, with that, it only shows time to time.

I have tested it again with help of p2p programs - connected to 10 dc hubs and started search, with setup similar to one above -> 100mbit/s internet <-wired link-> box with madwifi <-wireless link-> RB532 with 32MB - about 29 available, about 3MB free, min_free_kbytes default, which=756 there <- wired link-> windoze PC with p2p client software.

After i started search, i have watched athstats output and seen, that directly after starting search, count of out frames / in frames jums to over 1000 for one - three seconds - p2p is sending requests to hubs, opening connections to peers and closing them etc. . In this moment - if we are connected to more hubs and search for some frequently used word, like music or porn :), driver crash and connectivity between madwifi i386 box and RB532 is lost with above messages in syslog.

Apr 24 17:41:06 zmen_mne kernel: ath_uapsd_processtriggers: no skbuff
and stuff

From what i had read and understood, i think the following scenario come into place:

kernel allocates memory - conntrack entries, various slab buffers and caches, etc. - and deallocates memory very fast. It opens and closes TCP connections, which consumes memory. Inbetween, madwifi is jumping in need of another skbuffs. Sometimes, when memory is scarce, kernel throw the memory allocation failure. Then it take some time for it - kernel - to recover from this situation. But madwifi still needs skbuffs to be able to process incoming frames - i guess, will have to thoroughly study code related to this - and it found no available, so the

ath_uapsd_processtriggers: no skbuff

message is written into syslog and all is stopped . Nbd's NAPI patch make this scenario less frequent, but not impossible.

From what i have read on kernel mailing list, memory allocation failures happens time to time and are unavoidable, especially on low memory system. Suggested fix is to increase then min_free_kbytes. But, along with that, there is problem in madwifi that it will not recover automaticaly from this, rare, but unavoidable sometimes, situation. I will try to find solution for this and report - and post patch if any- progress soon.

news - i have sucessfully fixed this issue, but i think the fix is very evil and improper. Function ath_intr - interrupt handler - in case HAL_INT_RX is received normaly call ath_uapsd_processtrigers and then schedule ath_rx_tasklet . In case that there has been memory allocation failure, as is described above, i think system is in inconsistent state, some buffers are allocated, some not, so i put there in ath_uapsd_processtriggers - which is the first function which check for skbuff == NUL - the return call like this and changed this function to static int:

              skb = bf->bf_skb;
                if (skb == NULL) {              /* XXX ??? can this happen - sure, can*/
                        printk("%s: no skbuff, issuing reset\n", __func__);
                        alert = 1;
                        return alert;
                }

in ath_intr, i am testing return code of ath_uapsd_processtriggers, if it is not 0, global error counter is incremented and interupt ignored - simply not scheduling tasklet at all. In case kernel is recovered from memory allocation failure, ath_uapsd_processtriggers will not fail, thus return 0 and here is the problem for me. The code will tell all :

 if (status & HAL_INT_RX) {
                        sc->sc_tsf = ath_hal_gettsf64(ah);
                        if (ath_uapsd_processtriggers(sc))  {
                                    ecount++;
                                    printk("blah, ecount = %lu \n",ecount);
                        }
                        else {  if !( ecount 0 )
                                        ATH_SCHEDULE_TQUEUE(&sc->sc_rxtq, &needm
ark);
                                else {
                                        printk("in %s blahlaset\n",__func__);
                                        ath_hal_intrset(ah, 0);         /* disab
le intr's until reset */
                                        ATH_SCHEDULE_TQUEUE(&sc->sc_fataltq, &ne
edmark);
                                        ecount = 0;
                                }
                        }
                }

the problem is - is this at least somehow sane? It works, and i haven't been able to crash it yet, but i think that another IF added in interrupt handler will at least decrease performance, i think optimal way would be to do all this in ath_uapsd_processtriggers or even deeper - skbuff allocator? -, but i am out of ideas how to do it, because if i just try to issue reset there, it is disasterous - ecount is over 1000 sometimes so 1000 of resets and it will not work anyway at the end, which is something i still don't know why. Also, some received packets are ok, there is i think race with kernel for free memory, and thus some skbuffs are allocated in between. Some tasklets are scheduled, but most of them not during kernel recovery. At the end, without the reset, device is just silent, not receiving anything - i have seen several loops in logs when i added printks. Also, i have captured this crash - when driver is infested with printks, i managed to get this along with above pasted crashes :

[<8010956c>] dump_stack+0x8/0x34
[<8015e4b0>] __alloc_pages+0x2a0/0x348
[<8017f374>] cache_alloc_refill+0x340/0x7bc
[<8017fa48>] __kmalloc+0xd8/0x108
[<802f40b0>] __alloc_skb+0x68/0x134
[<c0185b3c>] ath_alloc_skb+0x2c/0x8c [ath_pci]
[<c0185d08>] ath_rxbuf_init+0x16c/0x238 [ath_pci]
[<c018ed58>] ath_rx_tasklet+0x1a4/0xd88 [ath_pci]
[<8012f458>] tasklet_action+0xc4/0x190
[<8012eda8>] __do_softirq+0x8c/0x134
[<8012eedc>] do_softirq+0x8c/0xb8
[<8012f900>] ksoftirqd+0x74/0xd4
[<801435bc>] kthread+0x284/0x2cc
[<801049f0>] kernel_thread_helper+0x10/0x18

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
Active:2517 inactive:2512 dirty:4 writeback:0 unstable:0 free:68 slab:1307 mapped:973 pagetables:38
DMA free:272kB min:720kB low:900kB high:1080kB active:10068kB inactive:10048kB present:32512kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0
DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 272kB
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 147160kB
Total swap = 147160kB
Free swap:       147160kB
8191 pages of RAM
0 pages of HIGHMEM
1180 reserved pages
3023 pages shared
0 pages swap cached
ath_rxbuf_init: skbuff alloc of size 3088 failed
ath_uapsd_processtriggers: no skbuff, issuing reset
ath_uapsd_processtriggers returned error, ignoring scheduling tasklet, ecount = 1 
ath_uapsd_processtriggers: no skbuff, issuing reset
ath_uapsd_processtriggers returned error, ignoring scheduling tasklet, ecount = 2 

Should we try to make ath_alloc_skb more robust? Is there feasible and sane way, to recover from kernel memory allocation failure in this function - for example defer allocation for some time? I will study this more, just want to report on what i am working and especially to get some hints, i think it is in skbuff allocator or in ath_rxbuff_init, but driver is pretty complicated for me on my current code reading skill level:)

after some more testing and studying code, this solution seems to me more and more improper, also it sometimes fails - when memory allocation failure in kernel happens in another stage of receiving process - for example in ath_startrecv. It will take a lot of time before some vital solution will be found i am afraid, i think there is something more fundamental which tends to break things when memory is low.

I think better way to deal with memory allocation failure would be - in ath_rxbuf_init - to

a) modify ath_alloc_skb to try again to allocate skbuff if first attempt fail and if another one fails, to deffer allocation - dunno if this concept is sane in driver code context, have to read more about this, it sound somehow wrong to me but haven't found better way to deal with this

b) create - for monitor mode handling - similar wrapper as ath_alloc_skb is, and not use dev_alloc_skb and company directly

will try to write and test code based on this and report back. I am sure there is some better way to deal with race for memory with other kernel parts and i plan to learn more to discover it

I've read similar reports of memory leaks in an OpenWrt? trac ticket. Somebody mentioned that using iwpriv ath0 protmode 0 makes the problem go away. Can you confirm this?

Never mind... It only made the problem go away for one guy and it's not related to this problem.

Unless this is a different memleak than the one that I'm getting, it is fixed by the following patch:

Index: madwifi-ng-r2420-20070602/net80211/ieee80211_input.c
===================================================================
--- madwifi-ng-r2420-20070602.orig/net80211/ieee80211_input.c	2007-07-02 12:50:31.839514500 +0200
+++ madwifi-ng-r2420-20070602/net80211/ieee80211_input.c	2007-07-02 12:51:10.309918750 +0200
@@ -2832,7 +2832,7 @@
 			ieee80211_add_scan(vap, &scan, wh, subtype, rssi, rstamp);
 			return;
 		}
-		if (scan.capinfo & IEEE80211_CAPINFO_IBSS) {
+		if ((vap->iv_opmode == IEEE80211_M_IBSS) && (scan.capinfo & IEEE80211_CAPINFO_IBSS)) {
 			if (!IEEE80211_ADDR_EQ(wh->i_addr2, ni->ni_macaddr)) {
 				/* Create a new entry in the neighbor table. */
 				ni = ieee80211_add_neighbor(vap, wh, &scan);

that's the place where I least expected this bug to be :)

Signed-off-by: Felix Fietkau <nbd@openwrt.org>

will try to test it in the evenning and report back

this patch is definitely not fully correct, but it shouldn't break anything important and it does fully plug the memleak in ap mode. in fact this patch probably just covers up a bug that is elsewhere (node refcounting?)

Applied as r2528. Testing welcome.

by me.

After night of testing it seems problem is away, but i am not yet sure, because i have tested it on madwifi with applied nbd's napi patch, which even without this fix made this problem hardly reproducible. Today, i will try to test unpatched trunk and report again.

So, problem is not away. I tried latest trunk with this patch applied and it has been perfectly reproducible again. So, probably NAPI patch again has somehow marginalized this.

Still, i am not sure problem here is in memleak, it seems in fact, that when madwifi is trying to allocate high order memory in time system is short on it - and allocation of it by system itself fails, althrought there are still plenty of memory available in fact, because when i set min_free_kbytes to some high value, it almost never occures and so there is memory just it is not clean and ready for fast allocation - it sometimes lost the race with other kernel stuff, mostly netfilter code - see traces.

This happens far more frequently when NAPI patch is not applied. I think we have to find reason why madwifi doesn't recover from this situation and stay stuck, till reset/ reassociation is forced somehow - either by scheduling reset tasklet - i tried to do it this way as i wrote before and it solved about 90% of causes, but NAPI patch alone seems to solve about 95% of them and is far more proper - or putting AP side of link - so not the one stuck - down and up again or whatever. Or, we could maybe use ring buffer as i have seen is used in another drivers, have to study it more though to be able to try to write something viable myself :) Traces from latest trunk follows :

root@zmen_mne:~# ksoftirqd/0: page allocation failure. order:0, mode:0x20
Call Trace:
[<8010956c>] dump_stack+0x8/0x34
[<8015e4b0>] __alloc_pages+0x2a0/0x348
[<8017f374>] cache_alloc_refill+0x340/0x7bc
[<8017fa48>] __kmalloc+0xd8/0x108
[<802f40b0>] __alloc_skb+0x68/0x134
[<c018cb2c>] ath_alloc_skb+0x2c/0x8c [ath_pci]
[<c018cd30>] ath_rxbuf_init+0x1a4/0x270 [ath_pci]
[<c01976c0>] ath_rx_tasklet+0x150/0xcd8 [ath_pci]
[<8012f458>] tasklet_action+0xc4/0x190
[<8012eda8>] __do_softirq+0x8c/0x134
[<8012eedc>] do_softirq+0x8c/0xb8
[<8012f900>] ksoftirqd+0x74/0xd4
[<801435bc>] kthread+0x284/0x2cc
[<801049f0>] kernel_thread_helper+0x10/0x18

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
Active:2821 inactive:1949 dirty:1 writeback:0 unstable:0 free:68 slab:1565 mapped:971 pagetables:38
DMA free:272kB min:720kB low:900kB high:1080kB active:11284kB inactive:7796kB present:32512kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0
DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 272kB
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 147160kB
Total swap = 147160kB
Free swap:       147160kB
8191 pages of RAM
0 pages of HIGHMEM
1180 reserved pages
3650 pages shared
0 pages swap cached
ksoftirqd/0: page allocation failure. order:0, mode:0x20
Call Trace:
[<8010956c>] dump_stack+0x8/0x34
[<8015e4b0>] __alloc_pages+0x2a0/0x348
[<8017f374>] cache_alloc_refill+0x340/0x7bc
[<8017f02c>] kmem_cache_alloc+0x90/0x98
[<80302c78>] dst_alloc+0x40/0x140
[<80326980>] ip_route_input_slow+0x9b4/0xe3c
[<8032700c>] ip_route_input+0x204/0x290
[<8032b664>] ip_rcv+0x3a8/0x7a4
[<802fe514>] netif_receive_skb+0x460/0x784
[<802fe94c>] process_backlog+0x114/0x2fc
[<802fec10>] net_rx_action+0xdc/0x2c0
[<8012eda8>] __do_softirq+0x8c/0x134
[<8012eedc>] do_softirq+0x8c/0xb8
[<8012f900>] ksoftirqd+0x74/0xd4
[<801435bc>] kthread+0x284/0x2cc
[<801049f0>] kernel_thread_helper+0x10/0x18

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
Active:2821 inactive:1949 dirty:1 writeback:0 unstable:0 free:68 slab:1565 mapped:971 pagetables:38
DMA free:272kB min:720kB low:900kB high:1080kB active:11284kB inactive:7796kB present:32512kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0
DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 272kB
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 147160kB
Total swap = 147160kB
Free swap:       147160kB
8191 pages of RAM
0 pages of HIGHMEM
1180 reserved pages
3650 pages shared
0 pages swap cached
ksoftirqd/0: page allocation failure. order:0, mode:0x20
Call Trace:
[<8010956c>] dump_stack+0x8/0x34
[<8015e4b0>] __alloc_pages+0x2a0/0x348
[<8017f374>] cache_alloc_refill+0x340/0x7bc
[<8017fa48>] __kmalloc+0xd8/0x108
[<802f40b0>] __alloc_skb+0x68/0x134
[<c018cb2c>] ath_alloc_skb+0x2c/0x8c [ath_pci]
[<c018cd30>] ath_rxbuf_init+0x1a4/0x270 [ath_pci]
[<c01976c0>] ath_rx_tasklet+0x150/0xcd8 [ath_pci]
[<8012f458>] tasklet_action+0xc4/0x190
[<8012eda8>] __do_softirq+0x8c/0x134
[<8012eedc>] do_softirq+0x8c/0xb8
[<8012f900>] ksoftirqd+0x74/0xd4
[<801435bc>] kthread+0x284/0x2cc
[<801049f0>] kernel_thread_helper+0x10/0x18

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
Active:2821 inactive:1949 dirty:1 writeback:0 unstable:0 free:68 slab:1565 mapped:971 pagetables:38
DMA free:272kB min:720kB low:900kB high:1080kB active:11284kB inactive:7796kB present:32512kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0
DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 272kB
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 147160kB
Total swap = 147160kB
Free swap:  

and from another run :

root@zmen_mne:~# kswapd0: page allocation failure. order:0, mode:0x20
Call Trace:
[<8010956c>] dump_stack+0x8/0x34
[<8015e4b0>] __alloc_pages+0x2a0/0x348
[<8017f374>] cache_alloc_refill+0x340/0x7bc
[<8017fa48>] __kmalloc+0xd8/0x108
[<802f40b0>] __alloc_skb+0x68/0x134
[<c018cb2c>] ath_alloc_skb+0x2c/0x8c [ath_pci]
[<c018cd30>] ath_rxbuf_init+0x1a4/0x270 [ath_pci]
[<c01976c0>] ath_rx_tasklet+0x150/0xcd8 [ath_pci]
[<8012f458>] tasklet_action+0xc4/0x190
[<8012eda8>] __do_softirq+0x8c/0x134
[<8012eedc>] do_softirq+0x8c/0xb8
[<80104278>] do_IRQ+0x24/0x34
[<80102880>] ret_from_irq+0x0/0x8
[<80165888>] shrink_zone+0xa88/0x1a2c
[<80166dc0>] kswapd+0x2d8/0x450
[<801435bc>] kthread+0x284/0x2cc
[<801049f0>] kernel_thread_helper+0x10/0x18

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
Active:2599 inactive:2280 dirty:1 writeback:0 unstable:0 free:68 slab:1424 mapped:970 pagetables:38
DMA free:272kB min:720kB low:900kB high:1080kB active:10396kB inactive:9120kB present:32512kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0
DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 272kB
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 147160kB
Total swap = 147160kB
Free swap:       147160kB
8191 pages of RAM
0 pages of HIGHMEM
1180 reserved pages
2990 pages shared
0 pages swap cached
kswapd0: page allocation failure. order:0, mode:0x20
Call Trace:
[<8010956c>] dump_stack+0x8/0x34
[<8015e4b0>] __alloc_pages+0x2a0/0x348
[<8017f374>] cache_alloc_refill+0x340/0x7bc
[<8017f02c>] kmem_cache_alloc+0x90/0x98
[<c01d3a9c>] ip_conntrack_alloc+0xa0/0x538 [ip_conntrack]
[<c01d42fc>] ip_conntrack_in+0x3c8/0x888 [ip_conntrack]
[<803207ac>] nf_iterate+0xbc/0x130
[<803208bc>] nf_hook_slow+0x9c/0x1a8
[<8032b518>] ip_rcv+0x25c/0x7a4
[<802fe514>] netif_receive_skb+0x460/0x784
[<802fe94c>] process_backlog+0x114/0x2fc
[<802fec10>] net_rx_action+0xdc/0x2c0
[<8012eda8>] __do_softirq+0x8c/0x134
[<8012eedc>] do_softirq+0x8c/0xb8
[<80104278>] do_IRQ+0x24/0x34
[<80102880>] ret_from_irq+0x0/0x8
[<80165888>] shrink_zone+0xa88/0x1a2c
[<80166dc0>] kswapd+0x2d8/0x450
[<801435bc>] kthread+0x284/0x2cc
[<801049f0>] kernel_thread_helper+0x10/0x18

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
Active:2599 inactive:2280 dirty:1 writeback:0 unstable:0 free:68 slab:1424 mapped:970 pagetables:38
DMA free:272kB min:720kB low:900kB high:1080kB active:10396kB inactive:9120kB present:32512kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0
DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 272kB
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 147160kB
Total swap = 147160kB
Free swap:       147160kB
8191 pages of RAM
0 pages of HIGHMEM
1180 reserved pages
2990 pages shared
0 pages swap cached
kswapd0: page allocation failure. order:0, mode:0x20
Call Trace:
[<8010956c>] dump_stack+0x8/0x34
[<8015e4b0>] __alloc_pages+0x2a0/0x348
[<8017f374>] cache_alloc_refill+0x340/0x7bc
[<8017fa48>] __kmalloc+0xd8/0x108
[<802f40b0>] __alloc_skb+0x68/0x134
[<c018cb2c>] ath_alloc_skb+0x2c/0x8c [ath_pci]
[<c018cd30>] ath_rxbuf_init+0x1a4/0x270 [ath_pci]
[<c01976c0>] ath_rx_tasklet+0x150/0xcd8 [ath_pci]
[<8012f458>] tasklet_action+0xc4/0x190
[<8012eda8>] __do_softirq+0x8c/0x134
[<8012eedc>] do_softirq+0x8c/0xb8
[<80104278>] do_IRQ+0x24/0x34
[<80102880>] ret_from_irq+0x0/0x8
[<80165888>] shrink_zone+0xa88/0x1a2c
[<80166dc0>] kswapd+0x2d8/0x450
[<801435bc>] kthread+0x284/0x2cc
[<801049f0>] kernel_thread_helper+0x10/0x18

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
Active:2599 inactive:2280 dirty:1 writeback:0 unstable:0 free:68 slab:1424 mapped:970 pagetables:38
DMA free:272kB min:720kB low:900kB high:1080kB active:10396kB inactive:9120kB present:32512kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0
DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 272kB
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 147160kB
Total swap = 147160kB
Free swap:       147160kB
8191 pages of RAM
0 pages of HIGHMEM
1180 reserved pages
2990 pages shared
0 pages swap cached
ath_uapsd_processtriggers: no skbuff
ath_uapsd_processtriggers: no skbuff

so there are two different questions - why madwifi stop to receive after this and doesn't recover somehow ? and, why this happens, is madwifi involved at all - i mean, is there memleak, does it consume too much memory or something, could there be more sane buffer allocation mechanism?

You can see the memory leak if you watch /proc/meminfo and see the value for 'Slab' increasing...

fixed in [2597]