question about AWE and /PAE

I have a question about the /3gb /pae switch that I was wondering if
someone could shed some light on.
My understanding is that when windows has a machine with 4GB of memory
2GB is reserved as usermode and 2gb is reserved for kernal mode. (This
is without using the /3gb switch)
My server has 5GB of memory. AWE is enabled and max server memory and
min server memory is set to 4125. I am using the /PAE switch but NOT
the /3GB
based on this the max memory that the SQL Server process shoudl have
access to is 3GB
2GB - Kernel Mode
2GB - User Mode
1GB - Access via AWE
But when I run perfmon and look at Total Server Memory this value
is 4125. How is this possible without using the /3GB switch?
Shouldn't this value be only 3GB?
BertWhat you see is correct. AWE does not require PAE or /3GB or any special OS
edition to do it's business.
PAE only helps it to see mem above 4 GB when available.
KB 274750 is the most correct source, I wouldn't trust anything else, there
are plenty of misleading publications about AWE :-)
Best practices would be another topic, but I am not certain that you have
the best config.
5GB gives you only 1 GB benefit vs 4GB + life without PAE-AWE, but you're
getting all the PAE-AWE overhead and bugs for this extra gig.
I'd pump memory up to 8GB to fully justify PAE-AWE, or sacrificed 1GB and
use /3GB only on a 4GB system if that doesn't hurt page life expectancy.
Regards,
AD
"bert" <bertcord@.gmail.com> wrote in message
news:1110229420.102441.138830@.z14g2000cwz.googlegroups.com...
> I have a question about the /3gb /pae switch that I was wondering if
> someone could shed some light on.
> My understanding is that when windows has a machine with 4GB of memory
> 2GB is reserved as usermode and 2gb is reserved for kernal mode. (This
> is without using the /3gb switch)
> My server has 5GB of memory. AWE is enabled and max server memory and
> min server memory is set to 4125. I am using the /PAE switch but NOT
> the /3GB
> based on this the max memory that the SQL Server process shoudl have
> access to is 3GB
> 2GB - Kernel Mode
> 2GB - User Mode
> 1GB - Access via AWE
> But when I run perfmon and look at Total Server Memory this value
> is 4125. How is this possible without using the /3GB switch?
> Shouldn't this value be only 3GB?
> Bert
>|||Thanks for your reply Andrew; I am familiar with the article. However
I still don't understand the interaction of the /3GB switch and the
/PAE switch
I started looking into this more after Brian Moran's article in SQL
magazine this month. Based on Brain's article the KB you mentioned
and several other resources on the net I have come up with the
following conclusions.
1. AWE is used to access memory over 4GB
2. To enabled AWE you must specify the /PAE switch in the boot.ini
3. on a 32 bit OS 4GB of memory can be directly accessed
4. a default windows install on a machine with 4GB will allocated 2GB
to Usermode memory and 2GB to kernel mode memory
5. you can change the kernel mode \user mode allocation by using the
3GB switch in the boot.ini, this will change it to 3GB Usermode 1GB
Kernel mode
The problem is based on these conclusions I don't understand how my
server is using 4125MB of memory as it is not using the /3GB switch.
Can anyone help see anything wrong with my points above or can
explanation of how my server is using 4125MB of memory?
Bert|||/PAE is an OS level switch that allows it to access more than 4GB of memory
by fooling it into thinking it is a 36 bit vs. 32 bit address space. The
/3GB is an app level (any app not just sql server) switch that lets the app
use up to 3GB (instead of the default of 2GB) of directly addressable ram.
Meaning the first 4GB. The reason SQL Server is using about 4GB is due to
the AWE setting which basically fools the app into thinking it is 36 bit
addressing as well. But in your case you are only using 2GB of directly
addressable memory and the other 2GB is thru AWE. Certain parts (most
actually) in sql servers memory allocation can only use the first 2GB (or
3GB if /3GB is turned on). The other 2GB is only used for the data buffer
cache. The buffer cache can also use memory below 2GB but the other parts
can not use above it. Hope that clears it up.
--
Andrew J. Kelly SQL MVP
"bert" <bertcord@.gmail.com> wrote in message
news:1110295182.235072.44080@.l41g2000cwc.googlegroups.com...
> Thanks for your reply Andrew; I am familiar with the article. However
> I still don't understand the interaction of the /3GB switch and the
> /PAE switch
> I started looking into this more after Brian Moran's article in SQL
> magazine this month. Based on Brain's article the KB you mentioned
> and several other resources on the net I have come up with the
> following conclusions.
> 1. AWE is used to access memory over 4GB
> 2. To enabled AWE you must specify the /PAE switch in the boot.ini
> 3. on a 32 bit OS 4GB of memory can be directly accessed
> 4. a default windows install on a machine with 4GB will allocated 2GB
> to Usermode memory and 2GB to kernel mode memory
> 5. you can change the kernel mode \user mode allocation by using the
> 3GB switch in the boot.ini, this will change it to 3GB Usermode 1GB
> Kernel mode
> The problem is based on these conclusions I don't understand how my
> server is using 4125MB of memory as it is not using the /3GB switch.
> Can anyone help see anything wrong with my points above or can
> explanation of how my server is using 4125MB of memory?
> Bert
>|||This article might help.
Inside SQL Server 2000's Memory Management Facilities
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnsqldev/html/sqldev_01262004.asp
"bert" <bertcord@.gmail.com> wrote in message
news:1110229420.102441.138830@.z14g2000cwz.googlegroups.com...
>I have a question about the /3gb /pae switch that I was wondering if
> someone could shed some light on.
> My understanding is that when windows has a machine with 4GB of memory
> 2GB is reserved as usermode and 2gb is reserved for kernal mode. (This
> is without using the /3gb switch)
> My server has 5GB of memory. AWE is enabled and max server memory and
> min server memory is set to 4125. I am using the /PAE switch but NOT
> the /3GB
> based on this the max memory that the SQL Server process shoudl have
> access to is 3GB
> 2GB - Kernel Mode
> 2GB - User Mode
> 1GB - Access via AWE
> But when I run perfmon and look at Total Server Memory this value
> is 4125. How is this possible without using the /3GB switch?
> Shouldn't this value be only 3GB?
> Bert
>|||I did some more reading on AWE this evening and I am wondering what
the advantage of using /3GB when also using /PAE.
According to books-on-line
"The memory pool of an instance of SQL Server 2000 using AWE cannot
be swapped out to the page file"
So in my server example
2GB - Locked in memory, accessed using AWE.
2GB - Is directly addressed. This memory is also locked and can not
be paged to disk.
Now if I enabled /3GB in the boot.ini it would change to as follows
1GB - Locked in memory, accessed using AWE.
3GB - Is directly addressed. Memory is also locked and can not be
paged to disk.
The one thing I am not 100% on is if the user mode section of memory is
also locked. IS this true due to the fact that the SQL Server is using
AWE?
Thanks
Bert|||> 1. AWE is used to access memory over 4GB
It is rather over 2GB for an application.
> 2. To enabled AWE you must specify the /PAE switch in the boot.ini
Not true.
You probably have many links telling opposite :-)
> 3. on a 32 bit OS 4GB of memory can be directly accessed
true
> 4. a default windows install on a machine with 4GB will allocated 2GB
> to Usermode memory and 2GB to kernel mode memory
true
> 5. you can change the kernel mode \user mode allocation by using the
> 3GB switch in the boot.ini, this will change it to 3GB Usermode 1GB
> Kernel mode
true. That's for direct access.
AWE can be used instead for a "cheat" method if app supports it.
> The problem is based on these conclusions I don't understand how my
> server is using 4125MB of memory as it is not using the /3GB switch.
It is using AWE instead|||You never want any memory that sql server uses to be paged to disk. That is
the whole point of sql server having it's own caching mechanism. If you
turn on AWE you need to ensure that you set your max memory setting to an
amount less than the total amount of memory so the OS always has access to
some.
--
Andrew J. Kelly SQL MVP
"bert" <bertcord@.gmail.com> wrote in message
news:1110342099.812640.6560@.f14g2000cwb.googlegroups.com...
>I did some more reading on AWE this evening and I am wondering what
> the advantage of using /3GB when also using /PAE.
> According to books-on-line
> "The memory pool of an instance of SQL Server 2000 using AWE cannot
> be swapped out to the page file"
> So in my server example
> 2GB - Locked in memory, accessed using AWE.
> 2GB - Is directly addressed. This memory is also locked and can not
> be paged to disk.
> Now if I enabled /3GB in the boot.ini it would change to as follows
> 1GB - Locked in memory, accessed using AWE.
> 3GB - Is directly addressed. Memory is also locked and can not be
> paged to disk.
> The one thing I am not 100% on is if the user mode section of memory is
> also locked. IS this true due to the fact that the SQL Server is using
> AWE?
> Thanks
> Bert
>|||I appreciate everyone trying to help me...but... my question is still
not answered. Maybe If I ask a different question it will help.
Ok lets take my example
1. SQL Server Enterprise on windows 2000 Advanced server
2. Server has 5GB of memory
3. /PAE is enabled
4. NO /3GB is used
5. AWE is enabled in SQL server
6. max server memory is not set
According to books online
"If max server memory has not been set and there is at least 3 GB of
free memory available on the computer, then the instance acquires
almost all of the available memory and may leave only up to 128 MB of
memory free."
Also from books-on-line
"Windows 2000 may require at least 1 GB of available memory to manage
AWE"
ok so in this example the server takes all memory but 128MB...... So
what about the second statement above? Where is the kernel mode memory
that windows need to use to manage the AWE memory? The server only has
128MB left...so I am assuming that it is paged out to disk.
I want to make sure that enough memory is left for the OS to manage the
AWE memory. The kernel mode section of memory manages this so you
should make sure SQL leaves enough physical memory so that the kernel
memory is not paged out to disk.
Thanks again
Bert|||The statement that it may take up to 1GB to manage AWE is hypothetical. If
you have upwards of 16GB and are using AWE the OS may actually need all 2GB
to operate properly. You always have to set a MAX memory when using AWE
otherwise SQL Server will allocate and lock all but 128MB and leave very
little for the OS. I never recommend leaving less than 1GB to start for the
OS.
--
Andrew J. Kelly SQL MVP
"bert" <bertcord@.gmail.com> wrote in message
news:1110381394.812121.45160@.z14g2000cwz.googlegroups.com...
>I appreciate everyone trying to help me...but... my question is still
> not answered. Maybe If I ask a different question it will help.
> Ok lets take my example
> 1. SQL Server Enterprise on windows 2000 Advanced server
> 2. Server has 5GB of memory
> 3. /PAE is enabled
> 4. NO /3GB is used
> 5. AWE is enabled in SQL server
> 6. max server memory is not set
> According to books online
> "If max server memory has not been set and there is at least 3 GB of
> free memory available on the computer, then the instance acquires
> almost all of the available memory and may leave only up to 128 MB of
> memory free."
> Also from books-on-line
> "Windows 2000 may require at least 1 GB of available memory to manage
> AWE"
> ok so in this example the server takes all memory but 128MB...... So
> what about the second statement above? Where is the kernel mode memory
> that windows need to use to manage the AWE memory? The server only has
> 128MB left...so I am assuming that it is paged out to disk.
> I want to make sure that enough memory is left for the OS to manage the
> AWE memory. The kernel mode section of memory manages this so you
> should make sure SQL leaves enough physical memory so that the kernel
> memory is not paged out to disk.
> Thanks again
> Bert
>|||Correct...
So with my server that I have with the max memory setting of 4125 and
not using the /3GB switch. There is a good chance that kernel mode
memory is being paged in and out.
This is due to the fact that SQL has 4GB locked, I only have 5GB and
the kernel mode address range is 2GB.
Has anyone performed and types of performance tests analyzing the use
of the /3GB boot.ini switch?
Bert|||Just because it can use up to 2GB does not mean it needs to. It will only
page if it needs more than the ~1GB you have left over for the OS. If your
OS is using more than that you have to find out why.
--
Andrew J. Kelly SQL MVP
"bert" <bertcord@.gmail.com> wrote in message
news:1110384098.888228.33880@.g14g2000cwa.googlegroups.com...
> Correct...
> So with my server that I have with the max memory setting of 4125 and
> not using the /3GB switch. There is a good chance that kernel mode
> memory is being paged in and out.
> This is due to the fact that SQL has 4GB locked, I only have 5GB and
> the kernel mode address range is 2GB.
> Has anyone performed and types of performance tests analyzing the use
> of the /3GB boot.ini switch?
> Bert
>|||Yes I agree...but the point I bring up is one that I have not seen
mentioned before.
When using AWE all memory that SQL server is using is locked in memory.
This includes the Bpool that resides in UserMode memory and the AWE
memory above 2GB or 3GB.
This is important because you need to make sure the OS has enough
physical memory to use for Kernalmode.
According to this link http://linktrim.com/1od
"On a computer running a member of the Windows Server 2003 family,
the Page Table area, which contains the Page Table Entries (PTEs), is
64 megabytes (MB) by default when 4GT is enabled. The Paged Pool space
is approximately 256 MB."
So in this case you would want to have at least 320MB of free physical
memory available so these sections are not paged out to disk. IT gets
more complicated when PAE is enabled. More PTEs are required to mange
this memory, so you're settling for MAX memory in SQL server should
be reduced.
I haven't tested this...not sure how to... but based on the above
facts I Would think it would be best to do as follows
1. If you are using /3GB leave 1GB free of available physical memory
for kernel mode
2. If you are not using /3GB leave 2GB free of available physical
memory for kernel mode.
Bert|||Yes that is true but you stated you had 5GB and the max memory set to about
4GB. That should leave plenty for the OS.
--
Andrew J. Kelly SQL MVP
"bert" <bertcord@.gmail.com> wrote in message
news:1110558787.002507.326490@.l41g2000cwc.googlegroups.com...
> Yes I agree...but the point I bring up is one that I have not seen
> mentioned before.
> When using AWE all memory that SQL server is using is locked in memory.
> This includes the Bpool that resides in UserMode memory and the AWE
> memory above 2GB or 3GB.
> This is important because you need to make sure the OS has enough
> physical memory to use for Kernalmode.
> According to this link http://linktrim.com/1od
> "On a computer running a member of the Windows Server 2003 family,
> the Page Table area, which contains the Page Table Entries (PTEs), is
> 64 megabytes (MB) by default when 4GT is enabled. The Paged Pool space
> is approximately 256 MB."
> So in this case you would want to have at least 320MB of free physical
> memory available so these sections are not paged out to disk. IT gets
> more complicated when PAE is enabled. More PTEs are required to mange
> this memory, so you're settling for MAX memory in SQL server should
> be reduced.
> I haven't tested this...not sure how to... but based on the above
> facts I Would think it would be best to do as follows
> 1. If you are using /3GB leave 1GB free of available physical memory
> for kernel mode
> 2. If you are not using /3GB leave 2GB free of available physical
> memory for kernel mode.
> Bert
>|||Andrew,
Do you know how much more meory the OS needs for PTE's based on a given
size of AWE memory. For example if my server is using say 3GB of AWE
memory how many additioanl MB of PTEs are required to manage this AWE
memory. IF this was know it migt be easier to determine how much
memory to leave free for the OS.
For example
64MB - Default PTE
256MB- Defaul Psged Pool space.
XXXMB - Additional PTE sapce to manange AWE
XXXMB - minimum amount of meory to leave
Thanks
Bert|||To the best of my knowledge it is not a straight calculation. It seems to
depend on how you are actually using the memory and how aggressive the AWE
re-mappings are. I have seen systems running up to 16GB with /3GB with
absolutely no problems. And I have also seen ones that have troubles at
12GB if the /3GB is set. 3GB of AWE should not be an issue at all. Now
there may be some nutty case out there where this is not true but it will be
the exception not the rule. The OS should operate properly on 1GB and
manage 3GB of AWE pretty easily. If you are running other apps on that
server they may play a part in overall memory management though.
Andrew J. Kelly SQL MVP
"bert" <bertcord@.gmail.com> wrote in message
news:1111160197.266230.224330@.f14g2000cwb.googlegroups.com...
> Andrew,
> Do you know how much more meory the OS needs for PTE's based on a given
> size of AWE memory. For example if my server is using say 3GB of AWE
> memory how many additioanl MB of PTEs are required to manage this AWE
> memory. IF this was know it migt be easier to determine how much
> memory to leave free for the OS.
> For example
> 64MB - Default PTE
> 256MB- Defaul Psged Pool space.
> XXXMB - Additional PTE sapce to manange AWE
> XXXMB - minimum amount of meory to leave
> Thanks
> Bert
>