Please note: This project is no longer active. The website is kept online for historic purposes only.
If you´re looking for a Linux driver for your Atheros WLAN device, you should continue here .

Ticket #2362 (new defect)

Opened 12 years ago

Last modified 11 years ago

soft lockup: spin lock conflict in madwifi r4100-20090929

Reported by: anonymous Assigned to:
Priority: major Milestone:
Component: madwifi: other Version: v0.9.4
Keywords: r4100 soft lockup Cc:
Patch is attached: 0 Pending: 0

Description

We've found a soft lockup issue with madwifi-r4100-20090929 when running hostapd. Kernel is 2.6.31.6, cards involved are Ubiquiti SR2 (AR5212). It occurs when domlme() triggers a transmit, the TX queue is locked, an interrupt occurs, the TX tasklet tries to lock the TX queue and a soft lockup occurs. It occurs randomly with a higher occurence under load, probably due to the increased chances of a domlme() triggered management frame being transmitted at the time the TX tasklet runs.

We have crash dumps from a number of AP's, but the backtrace is identical in all cases:

[19780.472004] BUG: soft lockup - CPU#0 stuck for 61s! [hostapd:2089]
[19780.472004] Modules linked in: wlan_ccmp(F) wlan_tkip(F) wlan_wep(F) wlan_xa
[19780.472004]
[19780.472004] Pid: 2089, comm: hostapd Tainted: PF          (2.6.31.6 #3)
[19780.472004] EIP: 0060:[<c174562e>] EFLAGS: 00000297 CPU: 0
[19780.472004] EIP is at _spin_lock+0x15/0x19
[19780.472004] EAX: df9a7828 EBX: 00000002 ECX: df9a7818 EDX: 00004a49
[19780.472004] ESI: df9a6340 EDI: df9a6340 EBP: dd275bf8 ESP: dd275bf8
[19780.472004]  DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
[19780.472004] CR0: 8005003b CR2: b77bc000 CR3: 1d223000 CR4: 000006d0
[19780.472004] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[19780.472004] DR6: ffff0ff0 DR7: 00000400
[19780.472004] Call Trace:
[19780.472004]  [<e0f2feab>] ath_tx_processq+0x98/0x61d [ath_pci]
[19780.472004]  [<c104da73>] ? sched_clock_cpu+0x128/0x132
[19780.472004]  [<c10d2833>] ? __getblk+0x1d/0x3e
[19780.472004]  [<e0f306d8>] ath_tx_tasklet+0x5a/0xd8 [ath_pci]
[19780.472004]  [<c103c2d9>] tasklet_action+0x5d/0x9a
[19780.472004]  [<c103bff4>] __do_softirq+0x9f/0x140
[19780.472004]  [<c103c13c>] irq_exit+0x2d/0x60
[19780.472004]  [<c10045d8>] do_IRQ+0x7d/0x93
[19780.472004]  [<c1003049>] common_interrupt+0x29/0x30
[19780.472004]  [<e0f2fd99>] ? ath_tx_start+0xcc8/0xd42 [ath_pci]
[19780.472004]  [<e0f2a679>] ath_mgtstart+0x1b9/0x226 [ath_pci]
[19780.472004]  [<e0e2ce3b>] ieee80211_mgmt_output+0x13a/0x142 [wlan]
[19780.472004]  [<e0e2f146>] ieee80211_send_mgmt+0xa42/0xa7c [wlan]
[19780.472004]  [<e0e36a4f>] domlme+0x2d/0x38 [wlan]
[19780.472004]  [<e0e36c22>] ieee80211_ioctl_setmlme+0x17d/0x22e [wlan]
[19780.472004]  [<c16b88f2>] ioctl_private_iw_point+0xa0/0xf1
[19780.472004]  [<c16b880c>] ? get_priv_descr_and_size+0x6a/0xb0
[19780.472004]  [<c16b8987>] ioctl_private_call+0x44/0x5b
[19780.472004]  [<e0e36aa5>] ? ieee80211_ioctl_setmlme+0x0/0x22e [wlan]
[19780.472004]  [<c16b8a2e>] wireless_process_ioctl+0x90/0xb5
[19780.472004]  [<e0e36aa5>] ? ieee80211_ioctl_setmlme+0x0/0x22e [wlan]
[19780.472004]  [<c16b8abe>] wext_ioctl_dispatch+0x3f/0x50
[19780.472004]  [<c16b8705>] ? ioctl_standard_call+0x0/0x9d
[19780.472004]  [<c16b8943>] ? ioctl_private_call+0x0/0x5b
[19780.472004]  [<c16b8af5>] wext_handle_ioctl+0x26/0x58
[19780.472004]  [<c16b8705>] ? ioctl_standard_call+0x0/0x9d
[19780.472004]  [<c16b8943>] ? ioctl_private_call+0x0/0x5b
[19780.472004]  [<c15fe9d0>] dev_ioctl+0x214/0x220
[19780.472004]  [<c15efeb8>] ? sock_ioctl+0x0/0x1ee
[19780.472004]  [<c15f009a>] sock_ioctl+0x1e2/0x1ee
[19780.472004]  [<c15efeb8>] ? sock_ioctl+0x0/0x1ee
[19780.472004]  [<c10c1a57>] vfs_ioctl+0x27/0x68
[19780.472004]  [<c10c24d3>] do_vfs_ioctl+0x174/0x17f
[19780.472004]  [<c10c251d>] sys_ioctl+0x3f/0x5a
[19780.472004]  [<c1002a65>] syscall_call+0x7/0xb
[19780.472004] Kernel panic - not syncing: softlockup: hung tasks
[19780.472004] Pid: 2089, comm: hostapd Tainted: PF          2.6.31.6 #3
[19780.472004] Call Trace:
[19780.472004]  [<c1036cf3>] panic+0x38/0xe2
[19780.472004]  [<c1072a24>] softlockup_tick+0x13d/0x145
[19780.472004]  [<c1040123>] run_local_timers+0x17/0x19
[19780.472004]  [<c103ff76>] update_process_times+0x24/0x4e
[19780.472004]  [<c1055239>] tick_sched_timer+0x6a/0x98
[19780.472004]  [<c104c333>] __run_hrtimer+0x5f/0x8c
[19780.472004]  [<c104c45b>] hrtimer_interrupt+0xfb/0x146
[19780.472004]  [<c101a381>] local_apic_timer_interrupt+0x42/0x47
[19780.472004]  [<c101a3b4>] smp_apic_timer_interrupt+0x2e/0x3d
[19780.472004]  [<c1003316>] apic_timer_interrupt+0x2a/0x30
[19780.472004]  [<c174562e>] ? _spin_lock+0x15/0x19
[19780.472004]  [<e0f2feab>] ath_tx_processq+0x98/0x61d [ath_pci]
[19780.472004]  [<c104da73>] ? sched_clock_cpu+0x128/0x132
[19780.472004]  [<c10d2833>] ? __getblk+0x1d/0x3e
[19780.472004]  [<e0f306d8>] ath_tx_tasklet+0x5a/0xd8 [ath_pci]
[19780.472004]  [<c103c2d9>] tasklet_action+0x5d/0x9a
[19780.472004]  [<c103bff4>] __do_softirq+0x9f/0x140
[19780.472004]  [<c103c0bb>] do_softirq+0x26/0x2b
[19780.472004]  [<c103c13c>] irq_exit+0x2d/0x60
[19780.472004]  [<c10045d8>] do_IRQ+0x7d/0x93
[19780.472004]  [<c1003049>] common_interrupt+0x29/0x30
[19780.472004]  [<e0f2fd99>] ? ath_tx_start+0xcc8/0xd42 [ath_pci]
[19780.472004]  [<e0f2a679>] ath_mgtstart+0x1b9/0x226 [ath_pci]
[19780.472004]  [<e0e2ce3b>] ieee80211_mgmt_output+0x13a/0x142 [wlan]
[19780.472004]  [<e0e2f146>] ieee80211_send_mgmt+0xa42/0xa7c [wlan]
[19780.472004]  [<e0e36a4f>] domlme+0x2d/0x38 [wlan]
[19780.472004]  [<e0e36c22>] ieee80211_ioctl_setmlme+0x17d/0x22e [wlan]
[19780.472004]  [<c16b88f2>] ioctl_private_iw_point+0xa0/0xf1
[19780.472004]  [<c16b880c>] ? get_priv_descr_and_size+0x6a/0xb0
[19780.472004]  [<c16b8987>] ioctl_private_call+0x44/0x5b
[19780.472004]  [<e0e36aa5>] ? ieee80211_ioctl_setmlme+0x0/0x22e [wlan]
[19780.472004]  [<c16b8a2e>] wireless_process_ioctl+0x90/0xb5
[19780.472004]  [<e0e36aa5>] ? ieee80211_ioctl_setmlme+0x0/0x22e [wlan]
[19780.472004]  [<c16b8abe>] wext_ioctl_dispatch+0x3f/0x50
[19780.472004]  [<c16b8705>] ? ioctl_standard_call+0x0/0x9d
[19780.472004]  [<c16b8943>] ? ioctl_private_call+0x0/0x5b
[19780.472004]  [<c16b8af5>] wext_handle_ioctl+0x26/0x58
[19780.472004]  [<c16b8705>] ? ioctl_standard_call+0x0/0x9d
[19780.472004]  [<c16b8943>] ? ioctl_private_call+0x0/0x5b
[19780.472004]  [<c15fe9d0>] dev_ioctl+0x214/0x220
[19780.472004]  [<c15efeb8>] ? sock_ioctl+0x0/0x1ee
[19780.472004]  [<c15f009a>] sock_ioctl+0x1e2/0x1ee
[19780.472004]  [<c15efeb8>] ? sock_ioctl+0x0/0x1ee
[19780.472004]  [<c10c1a57>] vfs_ioctl+0x27/0x68
[19780.472004]  [<c10c24d3>] do_vfs_ioctl+0x174/0x17f
[19780.472004]  [<c10c251d>] sys_ioctl+0x3f/0x5a
[19780.472004]  [<c1002a65>] syscall_call+0x7/0xb
[19780.472004] Rebooting in 10 seconds..

ath_tx_processq+0x98 resolves to the line marked below in ath/if_ath.c -> ath_tx_processq(). It occurs while trying to obtain the txq spinlock:

        for (;;) {
                if (uapsdq)
                        ATH_TXQ_UAPSDQ_LOCK_IRQ(txq);
                else
                        ATH_TXQ_LOCK(txq);  // ******* THIS LINE ********
                txq->axq_intrcnt = 0; /* reset periodic desc intr count */
                bf = STAILQ_FIRST(&txq->axq_q);

We seem to hit it approximately once every 2-6 hours on AP's under heavy load.

Change History

01/11/11 01:46:01 changed by peiweijun@21cn.com

I have the same, and using 2.6.26 at sta mode. Help!

01/11/11 08:44:20 changed by anonymous

The same problem, Help!!!