How to Analyze Deadlocked Schedulers Dumps?
Posted by Karthick P.K on June 15, 2010
How to Analyze "Deadlocked Schedulers" Dumps?
Do you see "Deadlocked Schedulers" errors similar to one below and stuck?
From SQL Server Errorlog
**Dump thread – spid = 0, PSS = 0×0000000000000000, EC = 0×0000000000000000
***Stack Dump being sent to C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\LOG\SQLDump0001.txt
* BEGIN STACK DUMP:
* Deadlocked Schedulers
* Short Stack Dump
Stack Signature for the dump is 0x00000000000003D0
New queries assigned to process on Node 0 have not been picked up by a worker thread in the last 60 seconds. Blocking or long-running queries can contribute to this condition, and may degrade client response time. Use the "max worker threads" configuration option to increase number of allowable threads, or optimize current running queries. SQL Process Utilization: 0%. System Idle: 69%.
New queries assigned to process on Node 3 have not been picked up by a worker thread in the last 300 seconds. Blocking or long-running queries can contribute to this condition, and may degrade client response time. Use the "max worker threads" configuration option to increase number of allowable threads, or optimize current running queries. SQL Process Utilization: 6%. System Idle: 90%.
Cause
We get Deadlocked Schedulers error (New queries assigned to process on Node n have not been picked up by a worker thread in the last 300 seconds) and dump when Scheduler Monitor detects threads(workers) are not Progressing on schedulers of one (or) all nodes for 60 seconds (300 seconds in Denali) and there are new work request (tasks) but there is no idle threads to pick up the the work request and SQL Server could not spawn new thread to process the new work request (Max worker threads reached).
Some of common causes are most of the tasks are waiting on a single resource because of resource bottleneck, excessive blocking, excessive parallelism, very long running Queries executed by all workers, all threads waiting on trace write waits etc..
SQL Server will not accept new connections or cannot process new requests when there is deadlocked scheduler condition (Exception: deadlocked condition on single node).
Note: In systems with multiple nodes (NUMA) If all the threads which belong to schedulers of single node is exhausted (or) Schedulers not progressing on single node can cause deadlocked scheduler condition.
Detailed explanation about dead lock schedulers can be found in This blog from my mentor J.P
Steps to analyze "Deadlocked Schedulers" Dumps.
To analyze the dump download and Install Windows Debugger from This link
Step 1:
Open Windbg . Choose File menu –> select Open crash dump –>Select the Dump file (SQLDump000#.mdmp)
Step 2:
on command window type
.sympath srv*c:\Websymbols*http://msdl.microsoft.com/download/symbols;
Step 3:
Type .reload /f and hit enter. This will force debugger to immediately load all the symbols.
Step 4:
Verify if symbols are loaded for SQL Server by using the debugger command lmvm
0:002> lmvm sqlservr
start end module name
00000000`01000000 00000000`03679000 sqlservr T (pdb symbols) c:\websymbols\sqlservr.pdb\21E4AC6E96294A529C9D99826B5A7C032\sqlservr.pdb
Loaded symbol image file: sqlservr.exe
Image path: C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Binn\sqlservr.exe
Image name: sqlservr.exe
Timestamp: Wed Oct 07 21:15:52 2009 (4ACD6778)
CheckSum: 025FEB5E
ImageSize: 02679000
File version: 2005.90.4266.0
Product version: 9.0.4266.0
File flags: 0 (Mask 3F)
File OS: 40000 NT Base
File type: 1.0 App
File date: 00000000.00000000
Translations: 0000.04b0 0000.04e4 0409.04b0 0409.04e4
Step 5:
Type ~*kL 20 and look at the stack of all the threads to find what majority of threads are doing.
Note: If you find most of your threads are waiting on stack which is not listed below please paste the stack in comments session of this blog (or) In MSSQLWIKI face book group we will try to answer you. If you don’t get prompt reply from the community, you may need to open a support ticket with Microsoft.
1. If it is blocking issue and If most of the threads are waiting to acquire a lock you will find the most of the stack similar to one below. (We try to acquire lock and go to wait, since someone is holding a lock)
ntdll!ZwSignalAndWaitForSingleObject
kernel32!SignalObjectAndWait
sqlservr!SOS_Scheduler::SwitchContext
sqlservr!SOS_Scheduler::Suspend
sqlservr!SOS_Event::Wait
sqlservr!LockOwner::Sleep
sqlservr!lck_lockInternal
sqlservr!GetLock
2. If most of threads are stuck while trying to write profiler events to the destination you might find stack similar to one below
ntdll!ZwSignalAndWaitForSingleObject
kernel32!SignalObjectAndWait
sqlservr!SOS_Scheduler::SwitchContext
sqlservr!SOS_Task::Sleep
sqlservr!CTraceRowsetIoProvider::GetFreeBuffers
sqlservr!CTraceWriteRequest::InitForRowsetTrace
sqlservr!CTraceRowsetIoProvider::InitializeWriteRequest
sqlservr!CTrace::WriteRecord
sqlservr!CTraceController::ProduceRecord
sqlservr!CTraceData::TracePreBatchEvent
sqlservr!CSQLSource::Execute
sqlservr!process_request
sqlservr!process_commands
sqlservr!SOS_Task::Param::Execute
sqlservr!SOS_Scheduler::RunTask
sqlservr!SOS_Scheduler::ProcessTasks
3. If your stack’s are like one below refer http://support.microsoft.com/default.aspx?scid=kb;EN-US;974205
sqlservr!SpinlockBase::Sleep sqlservr!SpinlockBase::SpinToAcquire sqlservr!TSyncHashTable_EntryAccessorsqlservr!CQSIndexStatsMgr::AddNewMissingIndex sqlservr!CIdxSuggestion::Register sqlservr!COptExpr::PqteConvert sqlservr!CPhyOp_Top::PqteConvert sqlservr!COptExpr::PqteConvert sqlservr!COptExpr::PqteConvertTree sqlservr!COptContext::PcxteOptimizeQuery sqlservr!CQuery::Optimize sqlservr!CQuery::PqoBuild sqlservr!CStmtQuery::InitQuery sqlservr!CStmtSelect::Init
4. If you see many stacks like the one below it could be BPOOL memory pressure (or) Lazy writer waiting on I/O
sqlservr!BPool::Steal
sqlservr!SQLSinglePageAllocator::AllocatePages
sqlservr!MemoryNode::AllocatePagesInternal
sqlservr!MemoryClerkInternal::AllocatePages
sqlservr!IMemObj::PbGetNewPages
sqlservr!CSlotPageMgr::PbAllocate
5. If you see many stacks like the one below it should be because of excessive parallelism
sqlservr!CQScanXProducerNew::Open
sqlservr!FnProducerOpen
sqlservr!FnProducerThread
sqlservr!SubprocEntrypoint
6. If you see many stacks like the one below (Many threads waiting to flush log) it should be because of disk bottleneck’s. Check if you see "I/O requests taking longer than 15 seconds" messages in Errorlog before Deadlocked Schedulers Dumps. Refer Troubleshooting sql server I/O for troubleshooting I/O issues.
sqlservr!SOS_Event::Wait
sqlservr!SQLServerLogMgr::WaitLCFlush
sqlservr!SQLServerLogMgr::LogFlush
sqlservr!SQLServerLogMgr::WaitLogFlush
sqlservr!XdesRMFull::Commit
If you liked this post, do like us on Facebook at https://www.facebook.com/mssqlwiki and join our Facebook group MSSQLWIKI to post your SQL Server questions to SQL Server experts
Related posts:
- How to analyze QL Server Exception , EXCEPTION_ACCESS_VIOLATION and SQL Server Assertion
- How to analyze Non-Yielding scheduler or Non-yielding IOCP Listener dumps ……
- Non-yielding IOCP Listener, Non-yielding Scheduler and non-yielding resource monitor known issues and fixes
- SQL Server generated Access Violation dumps while accessing oracle linked servers.
- SQL Server Latch & Debugging latch time out
Thank you,
Karthick P.K |My Facebook Page |My Site| Blog space| Twitter
Disclaimer
The views expressed on this website/blog are mine alone and do not reflect the views of my company. All postings on this blog are provided “AS IS” with no warranties, and confers no rights.
The Tao of the Deadlock Scheduler in SQL Server - SQL Meditation - Site Home - MSDN Blogs said
[...] Here are some examples of stacks that you will observe if you were to examine a memory dump (details on how to do so using public symbols can be found here) [...]
How to Analyze "Deadlocked Schedulers" Dumps? - Karthick PK 's Blog - Site Home - MSDN Blogs said
[...] Newer version of this post is available in http://mssqlwiki.com/2010/06/15/how-to-analyze-deadlocked-schedulers-dumps/ [...]
Xanax said
Hello, just wanted to tell you, I liked this post.
It was funny. Keep on posting!
Non-yielding IOCP Listener, Non-yielding Scheduler and non-yielding resource monitor known issues and fixes « MSSQLWIKI said
[...] How to Analyze "Deadlocked Schedulers" Dumps [...]
Nathan said
So general question on this thread. I wanted to review a dump file from a server issue last night. So I download WinDBG to my local PC, install it, copy the file from my production server, and then start following the steps listed above. Following along exactly as above, I get the following:
Loading Dump File [C:\Users\nathan.heaivilin\Desktop\DB9 Issue\SQLDump0074.mdmp]
User Mini Dump File: Only registers, stack and portions of memory are available
Comment: ‘Stack Trace’
Comment: ‘Deadlocked Schedulers’
Symbol search path is: *** Invalid ***
****************************************************************************
* Symbol loading may be unreliable without a symbol search path. *
* Use .symfix to have the debugger choose a symbol path. *
* After setting your symbol path, use .reload to refresh symbol locations. *
****************************************************************************
Executable search path is:
Windows Server 2003 Version 3790 (Service Pack 2) MP (8 procs) Free x64
Product: Server, suite: Enterprise TerminalServer SingleUserTS
Machine Name:
Debug session time: Thu Apr 4 04:50:50.000 2013 (UTC – 4:00)
System Uptime: not available
Process Uptime: 150 days 21:23:41.000
……………………………………………………….
……………………………………………
Loading unloaded module list
…………….
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(23fc.2358): Unknown exception – code 00000000 (first/second chance not available)
Unable to load image C:\WINDOWS\system32\ntdll.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for ntdll.dll
*** ERROR: Module load completed but symbols could not be loaded for ntdll.dll
ntdll+0x302ea:
00000000`77ef02ea c3 ret
0:004> .sympath srv*c:\Websymbols*http://msdl.microsoft.com/download/symbols;
Symbol search path is: srv*c:\Websymbols*http://msdl.microsoft.com/download/symbols
Expanded Symbol search path is: srv*c:\websymbols*http://msdl.microsoft.com/download/symbols
0:004> .reload /f
.Unable to load image C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Binn\sqlservr.exe, Win32 error 0n2
*** WARNING: Unable to verify timestamp for sqlservr.exe
………..
Press ctrl-c (cdb, kd, ntsd) or ctrl-break (windbg) to abort symbol loads that take too long.
Run !sym noisy before .reload to track down problems loading symbols.
…….Unable to load image C:\Program Files\Microsoft SQL Server\90\Shared\instapi.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for instapi.dll
*** ERROR: Module load completed but symbols could not be loaded for instapi.dll
.*** ERROR: Module load completed but symbols could not be loaded for sqlevn70.rll
………Unable to load image C:\WINDOWS\system32\iphlpapi.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for iphlpapi.dll
*** ERROR: Module load completed but symbols could not be loaded for iphlpapi.dll
..Unable to load image C:\WINDOWS\system32\msasn1.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for msasn1.dll
……………….Unable to load image C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Binn\msfte.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for msfte.dll
*** ERROR: Module load completed but symbols could not be loaded for msfte.dll
.Unable to load image C:\Program Files\Microsoft SQL Server\90\Shared\dbghelp.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for dbghelp.dll
.Unable to load image C:\WINDOWS\system32\wintrust.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for wintrust.dll
……..Unable to load image C:\WINDOWS\system32\xpsp2res.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for xpsp2res.dll
….
.Unable to load image C:\WINDOWS\system32\shlwapi.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for shlwapi.dll
…*** ERROR: Module load completed but symbols could not be loaded for sqlnclir.rll
.Unable to load image C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Binn\msftepxy.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for msftepxy.dll
*** ERROR: Module load completed but symbols could not be loaded for msftepxy.dll
……Unable to load image C:\WINDOWS\WinSxS\amd64_Microsoft.VC80.ATL_1fc8b3b9a1e18e3b_8.0.50727.42_x-ww_A08A3E21\ATL80.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for ATL80.dll
.Unable to load image C:\WINDOWS\system32\odbcint.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for odbcint.dll
*** ERROR: Module load completed but symbols could not be loaded for odbcint.dll
.*** ERROR: Module load completed but symbols could not be loaded for XPStar90.RLL
..*** ERROR: Module load completed but symbols could not be loaded for xplog70.rll
………Unable to load image C:\WINDOWS\assembly\NativeImages_v2.0.50727_64\mscorlibf50472cb7463465ca2c77c05167f5d8\mscorlib.ni.dll, Win32 error 0n2
*** WARNING: Unable to verify checksum for mscorlib.ni.dll
….Unable to load image C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Binn\SqlAccess.dll, Win32 error 0n2
…Unable to load image C:\WINDOWS\assembly\NativeImages_v2.0.50727_64\System\8965cd2ac127761b5ff62be938cbcf0c\System.ni.dll, Win32 error 0n2
*** WARNING: Unable to verify checksum for System.ni.dll
..Unable to load image C:\WINDOWS\assembly\NativeImages_v2.0.50727_64\System.Security\787d03a0028b5d45a70839dd6769a0d5\System.Security.ni.dll, Win32 error 0n2
*** WARNING: Unable to verify checksum for System.Security.ni.dll
..Unable to load image C:\WINDOWS\assembly\GAC_MSIL\System.Security\2.0.0.0__b03f5f7f11d50a3a\System.Security.dll, Win32 error 0n2
.Unable to load image C:\WINDOWS\assembly\GAC_MSIL\System.Xml\2.0.0.0__b77a5c561934e089\System.Xml.dll, Win32 error 0n2
.Unable to load image C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Binn\xpSLS.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for xpSLS.dll
*** ERROR: Module load completed but symbols could not be loaded for xpSLS.dll
.Unable to load image C:\Program Files\Microsoft SQL Server\80\COM\sqlvdi.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for sqlvdi.dll
………….
Loading unloaded module list
…………….
0:004> lmvm sqlserver
start end module name
Any thoughts on what I am doing wrong here:
My local PC is a Windows 7 64 bit install
Production server is a Windows Server 2003 64 bit install
SQL Server 2005 SP4 version
Thanks,
Nathan
Karthick P.K said
Hello Nathan,
It is lmvm sqlservr and not lmvm sqlserver
Nathan said
Karthick, thank you very much for the note. I was able to get that corrected. When redoing everything here is what I get.
0:004> .reload /f
.Unable to load image C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Binn\sqlservr.exe, Win32 error 0n2
*** WARNING: Unable to verify timestamp for sqlservr.exe
………………*** ERROR: Symbol file could not be found. Defaulted to export symbols for instapi.dll –
.*** ERROR: Module load completed but symbols could not be loaded for sqlevn70.rll
…….
Press ctrl-c (cdb, kd, ntsd) or ctrl-break (windbg) to abort symbol loads that take too long.
Run !sym noisy before .reload to track down problems loading symbols.
..*** ERROR: Symbol file could not be found. Defaulted to export symbols for iphlpapi.dll –
…………………*** ERROR: Symbol file could not be found. Defaulted to export symbols for msfte.dll –
…………..
….*** ERROR: Module load completed but symbols could not be loaded for sqlnclir.rll
.*** ERROR: Symbol file could not be found. Defaulted to export symbols for msftepxy.dll –
…….*** ERROR: Module load completed but symbols could not be loaded for odbcint.dll
.*** ERROR: Module load completed but symbols could not be loaded for XPStar90.RLL
..*** ERROR: Module load completed but symbols could not be loaded for xplog70.rll
………Unable to load image C:\WINDOWS\assembly\NativeImages_v2.0.50727_64\mscorlibf50472cb7463465ca2c77c05167f5d8\mscorlib.ni.dll, Win32 error 0n2
*** WARNING: Unable to verify checksum for mscorlib.ni.dll
….Unable to load image C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Binn\SqlAccess.dll, Win32 error 0n2
…Unable to load image C:\WINDOWS\assembly\NativeImages_v2.0.50727_64\System\8965cd2ac127761b5ff62be938cbcf0c\System.ni.dll, Win32 error 0n2
*** WARNING: Unable to verify checksum for System.ni.dll
..Unable to load image C:\WINDOWS\assembly\NativeImages_v2.0.50727_64\System.Security\787d03a0028b5d45a70839dd6769a0d5\System.Security.ni.dll, Win32 error 0n2
*** WARNING: Unable to verify checksum for System.Security.ni.dll
….Unable to load image C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Binn\xpSLS.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for xpSLS.dll
*** ERROR: Module load completed but symbols could not be loaded for xpSLS.dll
…………..
Loading unloaded module list
…………….
0:004> lmvm sqlservr
start end module name
00000000`01000000 00000000`0365d000 sqlservr T (pdb symbols) c:\websymbols\sqlservr.pdb\782EB41025744E27AB6DF074E02955B22\sqlservr.pdb
Loaded symbol image file: sqlservr.exe
Image path: C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Binn\sqlservr.exe
Image name: sqlservr.exe
Timestamp: Fri Mar 25 20:49:34 2011 (4D8D381E)
CheckSum: 025DF77F
ImageSize: 0265D000
File version: 2005.90.5057.0
Product version: 9.0.5057.0
File flags: 0 (Mask 3F)
File OS: 40000 NT Base
File type: 1.0 App
File date: 00000000.00000000
Translations: 0000.04b0 0000.04e4 0409.04b0 0409.04e4
So to me, everything is working as expected. However, when I run your ~*lK 20 command above, I get a ton of data, nothing of which had any of the items in red above.
Also, when running an !analze -v, I get the following.
0:004> !analyze -v
*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************
FAULTING_IP:
sqlservr!stackTrace+101
00000000`01f75c61 50 push rax
EXCEPTION_RECORD: ffffffffffffffff — (.exr 0xffffffffffffffff)
ExceptionAddress: 0000000001f75c61 (sqlservr!stackTrace+0×0000000000000101)
ExceptionCode: 00000000
ExceptionFlags: 00000000
NumberParameters: 0
DEFAULT_BUCKET_ID: WRONG_SYMBOLS
PROCESS_NAME: sqlservr.exe
MOD_LIST:
NTGLOBALFLAG: 0
MANAGED_STACK: !dumpstack -EE
OS Thread Id: 0×2358 (4)
Child-SP RetAddr Call Site
LAST_CONTROL_TRANSFER: from 0000000077d704ff to 0000000077ef02ea
FAULTING_THREAD: ffffffffffffffff
PRIMARY_PROBLEM_CLASS: WRONG_SYMBOLS
BUGCHECK_STR: APPLICATION_FAULT_WRONG_SYMBOLS
STACK_TEXT:
00000000`05ebdea8 00000000`77d704ff : 00000000`05ebe3e0 00000000`032e3ab0 00000000`0327fd28 00000000`05ebe2e0 : ntdll!ZwWaitForSingleObject+0xa
00000000`05ebdeb0 00000000`018cadd3 : 00000000`00000f98 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!WaitForSingleObjectEx+0×130
00000000`05ebdf50 00000000`018ca7cb : 00000000`0327fd28 00000000`0327fc40 00000000`00004e20 00000000`00000000 : sqlservr!CDmpDump::DumpInternal+0x4a3
00000000`05ebe040 00000000`01f72f93 : 00000000`0327fd28 00000000`03277c48 00000000`00010000 00000000`05ebe3e0 : sqlservr!CDmpDump::Dump+0x3b
00000000`05ebe090 00000000`01f76247 : 00000000`032e3ab0 00000000`00002358 00000000`00000331 00000000`05ebe3e0 : sqlservr!CImageHelper::DoMiniDump+0×413
00000000`05ebe1f0 00000000`02764ff3 : 00000000`00000000 00000000`01c259d9 00000000`00000000 00000000`000004a0 : sqlservr!stackTrace+0x6e7
00000000`05ebf720 00000000`01d4f300 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`05aa8080 : sqlservr!SQL_SOSDeadlockSchedulersCallback+0xa3
00000000`05ebf7a0 00000000`01b01986 : 00000000`00ba8088 00000000`00000001 00000000`00000001 00000000`00000000 : sqlservr!SOS_OS::ExecuteDeadlockSchedulerCallbacks+0×50
00000000`05ebf800 00000000`016b5cc7 : 00000000`206ab323 00000000`00000001 00000000`00ba9ae0 00000000`00000001 : sqlservr!alloca_probe+0x229d66
00000000`05ebf9a0 00000000`02b50c29 : 00000000`00ba9ae0 00000000`00000008 00000000`00000102 00000000`00000008 : sqlservr!SchedulerMonitor::Run+0×107
00000000`05ebfac0 00000000`0161ab6e : 00000000`00000001 00000000`00000000 00000000`8007a1c0 00000000`017ec7eb : sqlservr!SchedulerMonitor::EntryPoint+0×9
00000000`05ebfaf0 00000000`0161aec9 : 00000000`00000000 00000000`05aa8080 00000000`05aa4478 00000000`05aa8080 : sqlservr!SOS_Task::Param::Execute+0xee
00000000`05ebfc00 00000000`01611a94 : 00000000`05aa4478 00000000`00000000 ffffffff`fffffffe 00000000`05cb61c0 : sqlservr!SOS_Scheduler::RunTask+0xc9
00000000`05ebfc90 00000000`0179c057 : 00000000`05cb61c0 00000000`00ba8270 000007ff`fffad498 00000000`05aa8080 : sqlservr!SOS_Scheduler::ProcessTasks+0xb4
00000000`05ebfd00 00000000`014c6729 : 00000000`05cb61c0 00000000`05aa8080 0000170a`00000001 00000000`05ebfe18 : sqlservr!SchedulerManager::WorkerEntryPoint+0xe7
00000000`05ebfda0 00000000`017a1dc0 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`05ebfdc0 : sqlservr!SystemThread::RunWorker+0×59
00000000`05ebfde0 00000000`017ebdd8 : 00000000`00000000 00000000`00000000 00000000`0026f7d0 00000000`00000000 : sqlservr!SystemThreadDispatcher::ProcessWorker+0×130
00000000`05ebfe80 00000000`781337d7 : 000007ff`fffad498 00000000`005732f0 00000000`005732f0 00000000`005f0120 : sqlservr!SchedulerManager::ThreadEntryPoint+0×128
00000000`05ebff20 00000000`78133894 : 00000000`781e95c0 00000000`005732f0 00000000`00000000 00000000`00000000 : msvcr80!endthreadex+0×47
00000000`05ebff50 00000000`77d6b71a : 00000000`77d6b6e0 00000000`00000000 00000000`00000000 00000000`05ebffa8 : msvcr80!endthreadex+0×104
00000000`05ebff80 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadStart+0x3a
STACK_COMMAND: kb
FOLLOWUP_IP:
sqlservr!stackTrace+101
00000000`01f75c61 50 push rax
SYMBOL_STACK_INDEX: 0
SYMBOL_NAME: sqlservr!stackTrace+101
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: sqlservr
IMAGE_NAME: sqlservr.exe
DEBUG_FLR_IMAGE_TIMESTAMP: 4d8d381e
FAILURE_BUCKET_ID: WRONG_SYMBOLS_0_sqlservr.exe!stackTrace
BUCKET_ID: X64_APPLICATION_FAULT_WRONG_SYMBOLS_sqlservr!stackTrace+101
WATSON_STAGEONE_URL: http://watson.microsoft.com/StageOne/sqlservr_exe/2005_90_5057_0/4d8d381e/sqlservr_exe/2005_90_5057_0/4d8d381e/0/00f75c61.htm?Retriage=1
Followup: MachineOwner
———
Why is it still showing that I have the wrong sysmbols?
Thanks,
Nathan
Karthick P.K said
Nathan,
I got the output of ~*kL 20.
In most of the threads odbc32.dll is trying to acquire critical section. Looks like there is an orphaned critical section and other threads are waiting for that critical section to be released.
ntdll!ZwWaitForSingleObject+0xa
ntdll!RtlpWaitOnCriticalSection+0×240
ntdll!RtlEnterCriticalSection+0xa9
odbc32!CObjectPool::GetObjectFromPool+0xae
odbc32!MPInitializeCriticalSection+0x1b
odbc32!AllocEnv+0x11d
odbc32!SQLAllocHandle+0×190
sqlscm90!EnumServer+0×52
sqlscm90!IsSQLSvrClustered+0×98
sqlscm90!SQLSCMGetServiceStateNT+0x42d
sqlscm90!SQLSCMGetServiceStateW+0xb1
Apply the hotfix mentioned in http://support.microsoft.com/kb/940046 It should resolve the issue. If issue reoccurs after applying the patch please let me know.
Karthick P.K said
Nathan,
!analyze -v is only for exception dumps. We need output of ~*kL 20 as you mentioned output will be large so copy the the out put to text file and email me or upload file in our facebook group https://www.facebook.com/groups/454762937884205/
online graduate certificates said
Thank you for every other excellent post. The place else
may anyone get that type of information in such a perfect means of writing?
I have a presentation next week, and I’m on the look for such info.
PAGE NOT FOUND said
Hi there! I understand this is sort of off-topic but I had to ask.
Does managing a well-established blog like yours require a lot of
work? I am brand new to operating a blog but I do write in my journal everyday.
I’d like to start a blog so I can share my own experience and thoughts online. Please let me know if you have any kind of recommendations or tips for brand new aspiring blog owners. Thankyou!
nashville country music hall of fame museum said
Howdy! I simply wish to give an enormous thumbs up for
the good information you might have right here on this post.
I will be coming back to your blog for extra soon.
Dale said
After I originally left a comment I appear to have clicked on
the -Notify me when new comments are added- checkbox and
from now on whenever a comment is added I receive four emails with the exact same comment.
Perhaps there is an easy method you are able to remove me from
that service? Thanks!
british accounting review said
Hello! I simply wish to give a huge thumbs up for the
nice info you’ve got here on this post. I might be coming again to your weblog for more soon.
baratto said
Hi there, I found your website via Google at the same time as looking for
a comparable subject, your website got here up, it seems great.
I have bookmarked it in my google bookmarks.
Hi there, just turned into alert to your blog thru Google, and
found that it is truly informative. I am gonna be careful for brussels.
I’ll appreciate when you continue this in future. Many folks might be benefited from your writing. Cheers!
cognitive behavior therapist price said
0 and world-class Flickr Clone script site UI design. Anything that
you select will be stored in the correct drop down menu on the left side of your workspace.
com” domain name for your website for just $10 a year.
Bette said
This tree will give amazing feelings to her whenever she looks at the tree and see
how her family is grown up to a big and large family.
Total Beauty Collection’s Revlon Edition is only $15 (retail value is $30), plus free shipping. Skip the traditional bouquets, and try a unique bouquet from Fiesta Roses.
Arlie said
The article provides confirmed helpful to myself. It’s extremely useful and you are certainly really knowledgeable in
this field. You possess opened my own face in order
to different views on this specific subject matter
together with interesting and strong articles.
Cesar said
Hey There. I found your blog using msn. This is a really well written article.
I will be sure to bookmark it and come back to read more
of your useful information. Thanks for the post. I will definitely comeback.
zbot said
Hello would you mind letting me know which webhost you’re working with? I’ve loaded your blog in 3 different browsers and I must
say this blog loads a lot faster then most. Can you recommend a good internet hosting provider at a fair price?
Cheers, I appreciate it!
future shop uk said
Hello! I just wish to give an enormous thumbs up for the
good info you may have here on this post. I might be coming again to your blog for more soon.
Mariana said
A person essentially lend a hand to make severely articles
I might state. This is the very first time I
frequented your website page and to this point? I amazed with the analysis you made to create this particular post amazing.
Wonderful process!
buy Valium said
Your current report offers verified beneficial to us.
It’s quite informative and you are obviously very well-informed
in this region. You have exposed my face for you to numerous opinion of this particular topic along with intriquing, notable and reliable articles.
Microwave Ovens Who sells cheap said
Definitely believe that which you stated. Your favorite justification seemed to
be at the web the easiest thing to have in mind of.
I say to you, I definitely get irked at the same time as other people think about issues that they plainly do not recognise about.
You managed to hit the nail upon the top as well as outlined out the whole thing with no need side-effects ,
other people could take a signal. Will likely be back to get more.
Thank you
michael kors said
Hello to every one, because I am truly eager of reading this website’s post to be updated regularly. It contains pleasant material.
tv shows said
I think the admin of this web page is really working hard in favor of
his site, for the reason that here every data is quality based
information.
michael kors bag said
Hi there, I enjoy reading through your post. I like to write
a little comment to support you.
michael kors beverly fold-over clutch said
Thank you for the auspicious writeup. It in fact was a amusement account it.
Look advanced to more added agreeable from you!
By the way, how can we communicate?
vp finance jobs chicago said
Hey! I simply want to give a huge thumbs up for the nice information you will have right here on this post.
I will be coming back to your weblog for more soon.
timerdavid44.wordpress.com said
I have been surfing online more than 3 hours today, yet I never found any interesting article like yours.
It is pretty worth enough for me. In my opinion,
if all site owners and bloggers made good content as you did, the internet
will be much more useful than ever before.
plusonefoundry said
Hi there everyone, it’s my first visit at this site, and article is genuinely fruitful for me, keep up posting these articles.
plusone foundry said
I think the admin of this website is truly working hard for his web page, since here every stuff is quality based
information.
Beautifull Butt said
Hello! I’ve been following your website for a long time now and finally got the bravery to go ahead and give you a shout out from Dallas Texas! Just wanted to tell you keep up the good work!
schumacher 1200 a car battery jump starter said
Good day! I just wish to give an enormous thumbs up for the good info you have
got here on this post. I shall be coming back to your blog for more
soon.
chase home finance doraville ga said
Howdy! I just want to give an enormous thumbs up for the great information you have here on
this post. I will be coming back to your blog for extra
soon.