We are running 4 SQL Server 2000 servers with AWE enabled with 64 gb memory allocated, on windows 2003 R2 x64.
We are using -g Parameter 768 and we have activated the following traces at startup:
1204, 8599, 3601, 3603, 3940, 836.
We were running build 2282 since the year of 2009, but Microsoft recently released build 2305 and we applied it. Then a lot of memory problems started to show up.
We returned to build 2282 and the problem didn't show up anymore. We want to prevent vulnerabilities but we can't apply this hotfix.
We need help.
Here are the problems that show up in errorlog.
Problem 1
2012-09-28 20:04:36.12 spid95 WARNING: Failed to reserve contiguous memory of Size= 5373952.
2012-09-28 20:04:36.45 spid95 Buffer Distribution: Stolen=15582 Free=4038 Procedures=11615
Inram=0 Dirty=81639 Kept=0
I/O=321, Latched=448, Other=8205906
2012-09-28 20:04:36.45 spid95 Buffer Counts: Commited=8319549 Target=8319549 Hashed=8288014
InternalReservation=898 ExternalReservation=1378 Min Free=2712 Visible= 136416
2012-09-28 20:04:36.45 spid95 Procedure Cache: TotalProcs=4454 TotalPages=11615 InUsePages=4214
2012-09-28 20:04:36.45 spid95 Dynamic Memory Manager: Stolen=25513 OS Reserved=1264
OS Committed=1242
OS In Use=1132
Query Plan=10887 Optimizer=0
General=5207
Utilities=157 Connection=2130
2012-09-28 20:04:36.45 spid95 Global Memory Objects: Resource=2990 Locks=8405
SQLCache=357 Replication=2
LockBytes=2 ServerGlobal=48
Xact=192
2012-09-28 20:04:36.45 spid95 Query Memory Manager: Grants=5 Waiting=0 Maximum=78325 Available=75084
2012-09-28 20:04:36.46 spid95 Error: 17803, Severity: 20, State: 12
2012-09-28 20:04:36.46 spid95 Insufficient memory available..
Problem 2
2012-09-28 07:14:41.96 spid53 AWE mapping status:
2012-09-28 07:14:41.96 spid53 There are 70312 pages in the AWE window
2012-09-28 07:14:41.96 spid53 69594 pages have a bstat of 000000
2012-09-28 07:14:41.96 spid53 294 pages have a bstat of 000009
2012-09-28 07:14:41.96 spid53 288 pages have a bstat of 000020
2012-09-28 07:14:41.96 spid53 73 pages have a bstat of 00000b
2012-09-28 07:14:41.96 spid53 44 pages have a bstat of 000049
2012-09-28 07:14:41.96 spid53 11 pages have a bstat of 001001
2012-09-28 07:14:41.96 spid53 5 pages have a bstat of 00008b
2012-09-28 07:14:41.96 spid53 BPool::Map: no remappable address found.
2012-09-28 07:14:41.99 spid15 BPool::Map: no remappable address found.
2012-09-28 07:14:42.03 spid13 BPool::Map: no remappable address found.
2012-09-28 07:14:42.06 spid18 BPool::Map: no remappable address found.
2012-09-28 07:14:42.08 spid69 BPool::Map: no remappable address found.
2012-09-28 07:14:42.11 spid125 BPool::Map: no remappable address found.
2012-09-28 07:14:42.14 spid53 Buffer Distribution: Stolen=69550 Free=2086 Procedures=59
Inram=1147 Dirty=9567 Kept=0
I/O=0, Latched=150, Other=4111745
2012-09-28 07:14:42.14 spid53 Buffer Counts: Commited=4194304 Target=4194304 Hashed=4122609
InternalReservation=664 ExternalReservation=0 Min Free=2560 Visible= 70312
2012-09-28 07:14:42.14 spid53 Procedure Cache: TotalProcs=11 TotalPages=59 InUsePages=39
2012-09-28 07:14:42.14 spid53 Dynamic Memory Manager: Stolen=69606 OS Reserved=968
OS Committed=946
OS In Use=916
Query Plan=86 Optimizer=67296
General=2260
Utilities=22 Connection=356
2012-09-28 07:14:42.14 spid53 Global Memory Objects: Resource=1549 Locks=265
SQLCache=291 Replication=113
LockBytes=2 ServerGlobal=25
Xact=93
2012-09-28 07:14:42.14 spid53 Query Memory Manager: Grants=0 Waiting=0 Maximum=48 Available=48
2012-09-28 07:14:42.16 spid53 BPool::Map: no remappable address found.
2012-09-28 07:14:42.21 spid78 BPool::Map: no remappable address found.
2012-09-28 07:14:42.30 spid13 Buffer Distribution: Stolen=69551 Free=2085 Procedures=59
Inram=1147 Dirty=9567 Kept=0
I/O=0, Latched=150, Other=4111745
2012-09-28 07:14:42.30 spid13 Buffer Counts: Commited=4194304 Target=4194304 Hashed=4122609
InternalReservation=664 ExternalReservation=0 Min Free=2560 Visible= 70312
2012-09-28 07:14:42.30 spid13 Procedure Cache: TotalProcs=11 TotalPages=59 InUsePages=39
2012-09-28 07:14:42.30 spid13 Dynamic Memory Manager: Stolen=69605 OS Reserved=968
OS Committed=946
OS In Use=916
Query Plan=86 Optimizer=67295
General=2260
Utilities=22 Connection=356
2012-09-28 07:14:42.30 spid13 Global Memory Objects: Resource=1549 Locks=265
SQLCache=291 Replication=113
LockBytes=2 ServerGlobal=25
Xact=93
2012-09-28 07:14:42.30 spid13 Query Memory Manager: Grants=0 Waiting=0 Maximum=48 Available=48
2012-09-28 07:14:42.31 spid15 Buffer Distribution: Stolen=69550 Free=2086 Procedures=59
Inram=1147 Dirty=9567 Kept=0
I/O=0, Latched=150, Other=4111745
2012-09-28 07:14:42.31 spid15 Buffer Counts: Commited=4194304 Target=4194304 Hashed=4122609
InternalReservation=664 ExternalReservation=0 Min Free=2560 Visible= 70312
2012-09-28 07:14:42.31 spid15 Procedure Cache: TotalProcs=11 TotalPages=59 InUsePages=39
2012-09-28 07:14:42.31 spid15 Dynamic Memory Manager: Stolen=69605 OS Reserved=968
OS Committed=946
OS In Use=916
Query Plan=86 Optimizer=67295
General=2260
Utilities=22 Connection=356
2012-09-28 07:14:42.31 spid15 Global Memory Objects: Resource=1549 Locks=265
SQLCache=291 Replication=113
LockBytes=2 ServerGlobal=25
Xact=93
2012-09-28 07:14:42.31 spid15 Query Memory Manager: Grants=0 Waiting=0 Maximum=48 Available=48
2012-09-28 07:14:42.33 spid15 BPool::Map: no remappable address found.
2012-09-28 07:14:42.42 spid69 Buffer Distribution: Stolen=69550 Free=2086 Procedures=59
2012-09-28 07:14:42.42 spid18 Buffer Distribution: Stolen=69550 Free=2086 Procedures=59
Inram=1147 Dirty=9567 Kept=0
Inram=1147 Dirty=9567 Kept=0
I/O=0, Latched=149, Other=4111746
I/O=0, Latched=150, Other=4111745
2012-09-28 07:14:42.42 spid69 Buffer Counts: Commited=4194304 Target=4194304 Hashed=4122609
InternalReservation=664 ExternalReservation=0 Min Free=2560 Visible= 70312
2012-09-28 07:14:42.42 spid18 Buffer Counts: Commited=4194304 Target=4194304 Hashed=4122609
InternalReservation=664 ExternalReservation=0 Min Free=2560 Visible= 70312
2012-09-28 07:14:42.42 spid69 Procedure Cache: TotalProcs=11 TotalPages=59 InUsePages=39
2012-09-28 07:14:42.42 spid18 Procedure Cache: TotalProcs=11 TotalPages=59 InUsePages=39
2012-09-28 07:14:42.42 spid69 Dynamic Memory Manager: Stolen=69604 OS Reserved=968
OS Committed=946
OS In Use=916
Query Plan=86 Optimizer=67294
General=2260
Utilities=22 Connection=356
2012-09-28 07:14:42.42 spid18 Dynamic Memory Manager: Stolen=69604 OS Reserved=968
OS Committed=946
OS In Use=916
Query Plan=86 Optimizer=67294
General=2260
Utilities=22 Connection=356
2012-09-28 07:14:42.42 spid69 Global Memory Objects: Resource=1549 Locks=265
SQLCache=291 Replication=113
LockBytes=2 ServerGlobal=25
Xact=93
2012-09-28 07:14:42.42 spid18 Global Memory Objects: Resource=1549 Locks=265
SQLCache=291 Replication=113
LockBytes=2 ServerGlobal=25
Xact=93
2012-09-28 07:14:42.42 spid69 Query Memory Manager: Grants=0 Waiting=0 Maximum=48 Available=48
2012-09-28 07:14:42.42 spid18 Query Memory Manager: Grants=0 Waiting=0 Maximum=48 Available=48
2012-09-28 07:14:42.44 spid14 BPool::Map: no remappable address found.
2012-09-28 07:14:42.44 spid69 BPool::Map: no remappable address found.
2012-09-28 07:14:42.46 spid125 Buffer Distribution: Stolen=69551 Free=2085 Procedures=59
Inram=1147 Dirty=9567 Kept=0
I/O=0, Latched=149, Other=4111746
2012-09-28 07:14:42.46 spid125 Buffer Counts: Commited=4194304 Target=4194304 Hashed=4122609
InternalReservation=664 ExternalReservation=0 Min Free=2560 Visible= 70312
2012-09-28 07:14:42.46 spid125 Procedure Cache: TotalProcs=11 TotalPages=59 InUsePages=39
2012-09-28 07:14:42.46 spid125 Dynamic Memory Manager: Stolen=69605 OS Reserved=968
OS Committed=946
OS In Use=916
Query Plan=86 Optimizer=67295
General=2260
Utilities=22 Connection=356
2012-09-28 07:14:42.46 spid125 Global Memory Objects: Resource=1549 Locks=265
SQLCache=291 Replication=113
LockBytes=2 ServerGlobal=25
Xact=93
2012-09-28 07:14:42.46 spid125 Query Memory Manager: Grants=0 Waiting=0 Maximum=48 Available=48
2012-09-28 07:14:42.47 spid125 BPool::Map: no remappable address found.
2012-09-28 07:14:42.50 spid53 Buffer Distribution: Stolen=69078 Free=2568 Procedures=49
Inram=1147 Dirty=9567 Kept=0
I/O=0, Latched=149, Other=4111746
2012-09-28 07:14:42.50 spid53 Buffer Counts: Commited=4194304 Target=4194304 Hashed=4122135
InternalReservation=659 ExternalReservation=0 Min Free=2560 Visible= 70312
2012-09-28 07:14:42.50 spid53 Procedure Cache: TotalProcs=11 TotalPages=49 InUsePages=36
2012-09-28 07:14:42.50 spid53 Dynamic Memory Manager: Stolen=69596 OS Reserved=968
OS Committed=946
OS In Use=916
Query Plan=76 Optimizer=67296
General=2260
Utilities=22 Connection=356
2012-09-28 07:14:42.50 spid53 Global Memory Objects: Resource=1549 Locks=265
SQLCache=291 Replication=113
LockBytes=2 ServerGlobal=25
Xact=93
2012-09-28 07:14:42.50 spid53 Query Memory Manager: Grants=0 Waiting=0 Maximum=48 Available=48
2012-09-28 07:14:42.55 spid53 BPool::Map: no remappable address found.
2012-09-28 07:14:42.60 spid19 BPool::Map: no remappable address found.
2012-09-28 07:14:42.60 spid78 Buffer Distribution: Stolen=69082 Free=2564 Procedures=49
Inram=1147 Dirty=9567 Kept=0
I/O=0, Latched=149, Other=4111746
2012-09-28 07:14:42.60 spid78 Buffer Counts: Commited=4194304 Target=4194304 Hashed=4122135
InternalReservation=659 ExternalReservation=0 Min Free=2560 Visible= 70312
2012-09-28 07:14:42.60 spid78 Procedure Cache: TotalProcs=11 TotalPages=49 InUsePages=36
2012-09-28 07:14:42.60 spid78 Dynamic Memory Manager: Stolen=69601 OS Reserved=968
OS Committed=946
OS In Use=916
Query Plan=76 Optimizer=67301
General=2260
Utilities=22 Connection=356
2012-09-28 07:14:42.60 spid78 Global Memory Objects: Resource=1549 Locks=265
SQLCache=291 Replication=113
LockBytes=2 ServerGlobal=25
Xact=93