0.4 performance trouble
Locked
123
123
Re: 0.4 performance trouble
 
|
On 5 Apr 2007, at 12:49, James Page wrote: > > It feels to me like there may be a relatively simple "production > > only" bug happening > > Can you confirm if this only happening in Tomcat and not in Jetty? > Are you testing production in Tomcat. > > DT optimizations are writing to 'out' less times. I am wondering if > the Writer is blocking. The reason I feel this is you say that you > are getting the performance hit at different times, and different > places. 2 seconds is a very long time compared .6 seconds even if > the code is not optimized. > No, 2 requests per second, rather than 0.6/s but the same applies I imagine... Yes I think it might be blocking. It happens in production with Tomcat AND with Jetty locally. This is with concurrency tests, with 5 simultaneous clients and 1000 requests. > The quick thing to check also is that you are running java with the > server option -- just to check that it is not garbage collection, > getting in the way of your tests. Or even setting the gc collection > off for the test. > It's concurrency/load I'm sure. It is easily seen on the server. I run "top", request a single form page and CPU rockets to 100% for a few seconds before the page comes back - every time. That would concur with our observed (repeatedly) < 1 req/s for forms, but 22/s for normal content. > Could you send me the headers that your page is generating, or tell > me if the server is returning the CONTENT-LENGTH and the CHUNKED > header.. > I'll try to do this later. I've got to find some performance workarounds today and get them into the site or I'm going to get it in the neck. > The reason for looking at these headers is to see if the Server is > Writing out at the same time that you are Writing to the writer. In > the past I have seen many issues with Threads and Output streams. > OK, sounds interesting. I'm not ruling this out, but Grails does write everything out to a String/char array in memory first: static GSPResonseWriter getInstance(ServletResponse response, int max) { return new GSPResonseWriter(response, new CharArrayWriter(max), max); } [not my typo!] I'm playing with changing max currently. I notice the flush() is synchronized so even though there should be no contention for the lock, it will be acquired and flushed 3 times per request currently for me with the default buffer size and the page in question. > If it is the writer blocking there is code I can look at that I > wrote in the past that should get around this problem. > > I was going to write this later for our site, but if this is > causing issues now. I will write an implementation now, and post > it to the mailing list. > Thanks. I'm not so sure it is this. There are issues related to out.print, though it should be just writing to an in memory array. I am talking about 0.4.2 code by the way. Graeme has refactored GSP impl quite a bit in 0.5 but there are probably still things that can be done there. In 0.4.2 we do have several layers of buffering happening though. Each tag output is buffered to a StringWriter (with no default size set - this could be an issue for my large selects). The entire output of any GSP is dumped to a CharArrayWriter. Then it is flushed to the response's writer. THEN we have SiteMesh kicking in... Marc --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
Re: 0.4 performance trouble
|
In reply to this post by Marc Palmer Local
Marc Palmer wrote: > > > Yes, but please raise an issue for this Done http://jira.codehaus.org/browse/GRAILS-1024 > - we need to sort out the SetPropertiesDynamicMethod to not be > dailyWTF fodder. > Not sure it's possible is it? After all how do you know the context in which the call was made. That is to say if a user does "domainObject.properties" then you have to presume they need a properties map. Rather we need a Groovy (or Java) way to access a single property without throwing exception for those places where it's actually needed. > Marc > > > --------------------------------------------------------------------- > To unsubscribe from this list please visit: > > http://xircles.codehaus.org/manage_email > > --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
In reply to this post by Marc Palmer Local
Ok, I didn't know this was a concurrency-specific issue.
I definitely wonder about flush needing to be synchronized in both GSPWriter and GSPResonseWriter (man, that misspelling is annoying me!). PrintWriter already has a synchronized block inside of it (at least in 1.5) on the lock object: public void flush() { try { synchronized (lock) { ensureOpen(); out.flush(); } } catch (IOException x) { trouble = true; } } If really need to we can synchronize on the lock object which is a protected member from Writer. - Daiji Marc Palmer wrote: > > On 5 Apr 2007, at 12:49, James Page wrote: > >> > It feels to me like there may be a relatively simple "production >> > only" bug happening >> >> Can you confirm if this only happening in Tomcat and not in Jetty? >> Are you testing production in Tomcat. >> >> DT optimizations are writing to 'out' less times. I am wondering if >> the Writer is blocking. The reason I feel this is you say that you >> are getting the performance hit at different times, and different >> places. 2 seconds is a very long time compared .6 seconds even if the >> code is not optimized. >> > > No, 2 requests per second, rather than 0.6/s but the same applies I > imagine... > > Yes I think it might be blocking. It happens in production with Tomcat > AND with Jetty locally. This is with concurrency tests, with 5 > simultaneous clients and 1000 requests. > >> The quick thing to check also is that you are running java with the >> server option -- just to check that it is not garbage collection, >> getting in the way of your tests. Or even setting the gc collection >> off for the test. >> > > It's concurrency/load I'm sure. It is easily seen on the server. I run > "top", request a single form page and CPU rockets to 100% for a few > seconds before the page comes back - every time. That would concur > with our observed (repeatedly) < 1 req/s for forms, but 22/s for > normal content. > >> Could you send me the headers that your page is generating, or tell >> me if the server is returning the CONTENT-LENGTH and the CHUNKED >> header.. >> > > I'll try to do this later. I've got to find some performance > workarounds today and get them into the site or I'm going to get it in > the neck. > >> The reason for looking at these headers is to see if the Server is >> Writing out at the same time that you are Writing to the writer. In >> the past I have seen many issues with Threads and Output streams. >> > > OK, sounds interesting. I'm not ruling this out, but Grails does write > everything out to a String/char array in memory first: > > static GSPResonseWriter getInstance(ServletResponse response, int > max) { > return new GSPResonseWriter(response, new > CharArrayWriter(max), max); > } > > [not my typo!] > > I'm playing with changing max currently. I notice the flush() is > synchronized so even though there should be no contention for the > lock, it will be acquired and flushed 3 times per request currently > for me with the default buffer size and the page in question. > >> If it is the writer blocking there is code I can look at that I wrote >> in the past that should get around this problem. >> >> I was going to write this later for our site, but if this is causing >> issues now. I will write an implementation now, and post it to the >> mailing list. >> > > Thanks. I'm not so sure it is this. There are issues related to > out.print, though it should be just writing to an in memory array. > > I am talking about 0.4.2 code by the way. Graeme has refactored GSP > impl quite a bit in 0.5 but there are probably still things that can > be done there. > > In 0.4.2 we do have several layers of buffering happening though. Each > tag output is buffered to a StringWriter (with no default size set - > this could be an issue for my large selects). The entire output of any > GSP is dumped to a CharArrayWriter. Then it is flushed to the > response's writer. THEN we have SiteMesh kicking in... > > Marc > > > > --------------------------------------------------------------------- > To unsubscribe from this list please visit: > > http://xircles.codehaus.org/manage_email > > > > ____________________________________________________________________________________ Sucker-punch spam with award-winning protection. Try the free Yahoo! Mail Beta. http://advision.webevents.yahoo.com/mailbeta/features_spam.html --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
In reply to this post by Marc Palmer Local
On 4/5/07, Marc Palmer <[hidden email]> wrote:
> Could you send me the headers that your page is generating, or tell So if I gather right your from your earlier tests that the first test hits flush three times while the smaller test should only hit the close rather than the flush. My quess is that by increasing MAX by a factor of four or five you should get some performance increase, but maybe not the factor you are looking for. Maybe 10% 20%.... If it is more than that then it looks like a contention issue with the Writer. Your other pages are they smaller than 8024? What is GSPResonseWriter writing out to..... If you are not getting CONTENT-LENGTH coming back from the web server it suggests that there is another thread writing out your response while you are feeding it in. A test that you can see if this happening by looking at responce.isCommitted() if that is true then there could be a contention. Setting MAX to larger than 8k should cure the problem if it is contention. > If it is the writer blocking there is code I can look at that I It really smells to me like a contention /lock issue. But if you are running out of things to look at. Where is the data stored for countrySelect -- in the database? Is a new object been created for every country every time. If you reduce the size of the list of your country list does the problem go away. If you hard code it does it go away? |
|
|
In reply to this post by Marc Palmer Local
http://java.sun.com/j2se/1.4.2/docs/api/java/io/Writer.html#lock
"The object used to synchronize operations on this stream. For efficiency, a character-stream object may use an object other than itself to protect critical sections. A subclass should therefore use the object in this field rather than this or a synchronized method." D T wrote: > Ok, I didn't know this was a concurrency-specific issue. > > I definitely wonder about flush needing to be synchronized in both > GSPWriter and GSPResonseWriter (man, that misspelling is annoying me!). > PrintWriter already has a synchronized block inside of it (at least in > 1.5) on the lock object: > > public void flush() { > try { > synchronized (lock) { > ensureOpen(); > out.flush(); > } > } > catch (IOException x) { > trouble = true; > } > } > > If really need to we can synchronize on the lock object which is a > protected member from Writer. > > - Daiji > > Marc Palmer wrote: > >> On 5 Apr 2007, at 12:49, James Page wrote: >> >> >>>> It feels to me like there may be a relatively simple "production >>>> only" bug happening >>>> >>> Can you confirm if this only happening in Tomcat and not in Jetty? >>> Are you testing production in Tomcat. >>> >>> DT optimizations are writing to 'out' less times. I am wondering if >>> the Writer is blocking. The reason I feel this is you say that you >>> are getting the performance hit at different times, and different >>> places. 2 seconds is a very long time compared .6 seconds even if the >>> code is not optimized. >>> >>> >> No, 2 requests per second, rather than 0.6/s but the same applies I >> imagine... >> >> Yes I think it might be blocking. It happens in production with Tomcat >> AND with Jetty locally. This is with concurrency tests, with 5 >> simultaneous clients and 1000 requests. >> >> >>> The quick thing to check also is that you are running java with the >>> server option -- just to check that it is not garbage collection, >>> getting in the way of your tests. Or even setting the gc collection >>> off for the test. >>> >>> >> It's concurrency/load I'm sure. It is easily seen on the server. I run >> "top", request a single form page and CPU rockets to 100% for a few >> seconds before the page comes back - every time. That would concur >> with our observed (repeatedly) < 1 req/s for forms, but 22/s for >> normal content. >> >> >>> Could you send me the headers that your page is generating, or tell >>> me if the server is returning the CONTENT-LENGTH and the CHUNKED >>> header.. >>> >>> >> I'll try to do this later. I've got to find some performance >> workarounds today and get them into the site or I'm going to get it in >> the neck. >> >> >>> The reason for looking at these headers is to see if the Server is >>> Writing out at the same time that you are Writing to the writer. In >>> the past I have seen many issues with Threads and Output streams. >>> >>> >> OK, sounds interesting. I'm not ruling this out, but Grails does write >> everything out to a String/char array in memory first: >> >> static GSPResonseWriter getInstance(ServletResponse response, int >> max) { >> return new GSPResonseWriter(response, new >> CharArrayWriter(max), max); >> } >> >> [not my typo!] >> >> I'm playing with changing max currently. I notice the flush() is >> synchronized so even though there should be no contention for the >> lock, it will be acquired and flushed 3 times per request currently >> for me with the default buffer size and the page in question. >> >> >>> If it is the writer blocking there is code I can look at that I wrote >>> in the past that should get around this problem. >>> >>> I was going to write this later for our site, but if this is causing >>> issues now. I will write an implementation now, and post it to the >>> mailing list. >>> >>> >> Thanks. I'm not so sure it is this. There are issues related to >> out.print, though it should be just writing to an in memory array. >> >> I am talking about 0.4.2 code by the way. Graeme has refactored GSP >> impl quite a bit in 0.5 but there are probably still things that can >> be done there. >> >> In 0.4.2 we do have several layers of buffering happening though. Each >> tag output is buffered to a StringWriter (with no default size set - >> this could be an issue for my large selects). The entire output of any >> GSP is dumped to a CharArrayWriter. Then it is flushed to the >> response's writer. THEN we have SiteMesh kicking in... >> >> Marc >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe from this list please visit: >> >> http://xircles.codehaus.org/manage_email >> >> >> >> >> > > > > > ____________________________________________________________________________________ > Sucker-punch spam with award-winning protection. > Try the free Yahoo! Mail Beta. > http://advision.webevents.yahoo.com/mailbeta/features_spam.html > > --------------------------------------------------------------------- > To unsubscribe from this list please visit: > > http://xircles.codehaus.org/manage_email > > > > > ____________________________________________________________________________________ We won't tell. Get more on shows you hate to love (and love to hate): Yahoo! TV's Guilty Pleasures list. http://tv.yahoo.com/collections/265 --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
Re: 0.4 performance trouble
|
|
In reply to this post by Maurice Nicholson old
On 5 Apr 2007, at 13:32, Maurice Nicholson wrote: > > > Marc Palmer wrote: >> >> >> Yes, but please raise an issue for this > > Done http://jira.codehaus.org/browse/GRAILS-1024 > >> - we need to sort out the SetPropertiesDynamicMethod to not be >> dailyWTF fodder. >> > > Not sure it's possible is it? After all how do you know the context > in which the call was made. That is to say if a user does > "domainObject.properties" then you have to presume they need a > properties map. > > Rather we need a Groovy (or Java) way to access a single property > without throwing exception for those places where it's actually > needed. I don't see why we can't return a proxy map. Marc --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
In reply to this post by Marc Palmer Local
There's also a question about how we instantiate CharArrayWriter from
GSPResonseWriter using the default constructor, which is: public CharArrayWriter() { this(32); } And it resizes by a factor of 2 every time it needs to do so. As for StringWriter (for tag invocations), it's even smaller: public StringWriter() { buf = new StringBuffer(); lock = buf; } public StringBuffer() { super(16); } - Daiji D T wrote: > http://java.sun.com/j2se/1.4.2/docs/api/java/io/Writer.html#lock > > "The object used to synchronize operations on this stream. For > efficiency, a character-stream object may use an object other than > itself to protect critical sections. A subclass should therefore use the > object in this field rather than this or a synchronized method." > > D T wrote: > >> Ok, I didn't know this was a concurrency-specific issue. >> >> I definitely wonder about flush needing to be synchronized in both >> GSPWriter and GSPResonseWriter (man, that misspelling is annoying me!). >> PrintWriter already has a synchronized block inside of it (at least in >> 1.5) on the lock object: >> >> public void flush() { >> try { >> synchronized (lock) { >> ensureOpen(); >> out.flush(); >> } >> } >> catch (IOException x) { >> trouble = true; >> } >> } >> >> If really need to we can synchronize on the lock object which is a >> protected member from Writer. >> >> - Daiji >> >> Marc Palmer wrote: >> >> >>> On 5 Apr 2007, at 12:49, James Page wrote: >>> >>> >>> >>>>> It feels to me like there may be a relatively simple "production >>>>> only" bug happening >>>>> >>>>> >>>> Can you confirm if this only happening in Tomcat and not in Jetty? >>>> Are you testing production in Tomcat. >>>> >>>> DT optimizations are writing to 'out' less times. I am wondering if >>>> the Writer is blocking. The reason I feel this is you say that you >>>> are getting the performance hit at different times, and different >>>> places. 2 seconds is a very long time compared .6 seconds even if the >>>> code is not optimized. >>>> >>>> >>>> >>> No, 2 requests per second, rather than 0.6/s but the same applies I >>> imagine... >>> >>> Yes I think it might be blocking. It happens in production with Tomcat >>> AND with Jetty locally. This is with concurrency tests, with 5 >>> simultaneous clients and 1000 requests. >>> >>> >>> >>>> The quick thing to check also is that you are running java with the >>>> server option -- just to check that it is not garbage collection, >>>> getting in the way of your tests. Or even setting the gc collection >>>> off for the test. >>>> >>>> >>>> >>> It's concurrency/load I'm sure. It is easily seen on the server. I run >>> "top", request a single form page and CPU rockets to 100% for a few >>> seconds before the page comes back - every time. That would concur >>> with our observed (repeatedly) < 1 req/s for forms, but 22/s for >>> normal content. >>> >>> >>> >>>> Could you send me the headers that your page is generating, or tell >>>> me if the server is returning the CONTENT-LENGTH and the CHUNKED >>>> header.. >>>> >>>> >>>> >>> I'll try to do this later. I've got to find some performance >>> workarounds today and get them into the site or I'm going to get it in >>> the neck. >>> >>> >>> >>>> The reason for looking at these headers is to see if the Server is >>>> Writing out at the same time that you are Writing to the writer. In >>>> the past I have seen many issues with Threads and Output streams. >>>> >>>> >>>> >>> OK, sounds interesting. I'm not ruling this out, but Grails does write >>> everything out to a String/char array in memory first: >>> >>> static GSPResonseWriter getInstance(ServletResponse response, int >>> max) { >>> return new GSPResonseWriter(response, new >>> CharArrayWriter(max), max); >>> } >>> >>> [not my typo!] >>> >>> I'm playing with changing max currently. I notice the flush() is >>> synchronized so even though there should be no contention for the >>> lock, it will be acquired and flushed 3 times per request currently >>> for me with the default buffer size and the page in question. >>> >>> >>> >>>> If it is the writer blocking there is code I can look at that I wrote >>>> in the past that should get around this problem. >>>> >>>> I was going to write this later for our site, but if this is causing >>>> issues now. I will write an implementation now, and post it to the >>>> mailing list. >>>> >>>> >>>> >>> Thanks. I'm not so sure it is this. There are issues related to >>> out.print, though it should be just writing to an in memory array. >>> >>> I am talking about 0.4.2 code by the way. Graeme has refactored GSP >>> impl quite a bit in 0.5 but there are probably still things that can >>> be done there. >>> >>> In 0.4.2 we do have several layers of buffering happening though. Each >>> tag output is buffered to a StringWriter (with no default size set - >>> this could be an issue for my large selects). The entire output of any >>> GSP is dumped to a CharArrayWriter. Then it is flushed to the >>> response's writer. THEN we have SiteMesh kicking in... >>> >>> Marc >>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe from this list please visit: >>> >>> http://xircles.codehaus.org/manage_email >>> >>> >>> >>> >>> >>> >> >> >> ____________________________________________________________________________________ >> Sucker-punch spam with award-winning protection. >> Try the free Yahoo! Mail Beta. >> http://advision.webevents.yahoo.com/mailbeta/features_spam.html >> >> --------------------------------------------------------------------- >> To unsubscribe from this list please visit: >> >> http://xircles.codehaus.org/manage_email >> >> >> >> >> >> > > > > > ____________________________________________________________________________________ > We won't tell. Get more on shows you hate to love > (and love to hate): Yahoo! TV's Guilty Pleasures list. > http://tv.yahoo.com/collections/265 > > --------------------------------------------------------------------- > To unsubscribe from this list please visit: > > http://xircles.codehaus.org/manage_email > > > > > ____________________________________________________________________________________ Don't pick lemons. See all the new 2007 cars at Yahoo! Autos. http://autos.yahoo.com/new_cars.html --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
Re: 0.4 performance trouble
|
|
In reply to this post by JamesPage
On 5 Apr 2007, at 14:36, James Page wrote: > > So if I gather right your from your earlier tests that the first > test hits flush three times while the smaller test should only hit > the close rather than the flush. My quess is that by increasing MAX > by a factor of four or five you should get some performance > increase, but maybe not the factor you are looking for. Maybe 10% > 20%.... If it is more than that then it looks like a contention > issue with the Writer. > > Your other pages are they smaller than 8024? > > What is GSPResonseWriter writing out to..... If you are not getting > CONTENT-LENGTH coming back from the web server it suggests that > there is another thread writing out your response while you are > feeding it in. A test that you can see if this happening by looking > at responce.isCommitted() if that is true then there could be a > contention. > > Setting MAX to larger than 8k should cure the problem if it is > contention. > I also tried caching the per-tag invocation StringWriter for the duration of the request (per page) and this was slower than just new StringWriter() every time (yes I tried larger init sizes too). In this latter case I think it shows that storing a request attribute is a relatively expensive process compared to new StringWriter() and the buffer expansions that occur as it is written to. Yes the home page is < 8k. I will try killing the sync on flush() but I doubt this will make much difference. OK - killing the sync made no noticeable difference, which reinforces my belief that with modern VMs that uncontended syncs are not a major concern (there is only one thread writing to the output after all). However I have got a 10-15% improved by taking out the code in GSPResonseWriter.invokeMethod that relies on exceptions from super.invokeMethod to identify tags to execute. This is a dirty hack as it breaks method calls on the Script representing the GSP, but I don't -think- I use this anywhere. I've also seen gains from using a single GString in some places in g:select's impl and g:datePicker, and calling HtmlEscape directly instead of x.encodeAsHTML() For the latter, I don't believe it is inherently slow to use EMC, but the current CodecsPlugin impl is having to get the codec every time from a Map (not in itself particularly expensive) because of reloading - the codecs can be reloaded and hence could change, and therefore the codec methods cannot keep a reference to the codec class. I don't think you can play with the EMC methods after doWithDynamicMethods as I think initialize() is called and then it's locked down. However it would be trivial to add code to the codecs plugin that checks the environment, and only if it is development make it look up the codec - else hardwire it in there. I will try that in a moment to see if it helps with some of the other tags in the form. Marc --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
Re: 0.4 performance trouble
|
|
In reply to this post by D T-2
On 5 Apr 2007, at 14:50, D T wrote: > There's also a question about how we instantiate CharArrayWriter from > GSPResonseWriter using the default constructor, which is: > > public CharArrayWriter() { > this(32); > } > > And it resizes by a factor of 2 every time it needs to do so. > > As for StringWriter (for tag invocations), it's even smaller: > > public StringWriter() { > buf = new StringBuffer(); > lock = buf; > } > > public StringBuffer() { > super(16); > } I'm a bit more concerned by the copying of the entire response String to a char array with toCharArray() so it can be written to the CharArrayWriter. Surely this is a new copy of the string as an array... in my case 26KB, which equals 26000 loop iterations, albeit with a JNI'd System.arraycopy I ... hope... Marc --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
On 4/5/07, Marc Palmer <[hidden email]> wrote:
Does 'out' not get re-instantiate by the constructor which sets it length to 8024 bytes. > |
|
|
In reply to this post by Marc Palmer Local
Another thought. Can we
1) pool the Writer/Closure instances used for calling tags (so we're not constructing/GC'ing so much)? 2) implement a Writer that takes a CharSequence parameter in its constructor, and implement a custom CharSequence that will nest other CharSequence objects, rather than trying to grow itself by making copies into a larger and larger array? I suspect that those two steps alone will make a clear difference in the runtime of a page with lots of nested tags and templates. But maybe this is stuff for 0.5 / 0.6, as it's probably not trivial stuff to implement. - Daiji D T wrote: > There's also a question about how we instantiate CharArrayWriter from > GSPResonseWriter using the default constructor, which is: > > public CharArrayWriter() { > this(32); > } > > And it resizes by a factor of 2 every time it needs to do so. > > As for StringWriter (for tag invocations), it's even smaller: > > public StringWriter() { > buf = new StringBuffer(); > lock = buf; > } > > public StringBuffer() { > super(16); > } > > - Daiji > > D T wrote: > >> http://java.sun.com/j2se/1.4.2/docs/api/java/io/Writer.html#lock >> >> "The object used to synchronize operations on this stream. For >> efficiency, a character-stream object may use an object other than >> itself to protect critical sections. A subclass should therefore use the >> object in this field rather than this or a synchronized method." >> >> D T wrote: >> >> >>> Ok, I didn't know this was a concurrency-specific issue. >>> >>> I definitely wonder about flush needing to be synchronized in both >>> GSPWriter and GSPResonseWriter (man, that misspelling is annoying me!). >>> PrintWriter already has a synchronized block inside of it (at least in >>> 1.5) on the lock object: >>> >>> public void flush() { >>> try { >>> synchronized (lock) { >>> ensureOpen(); >>> out.flush(); >>> } >>> } >>> catch (IOException x) { >>> trouble = true; >>> } >>> } >>> >>> If really need to we can synchronize on the lock object which is a >>> protected member from Writer. >>> >>> - Daiji >>> >>> Marc Palmer wrote: >>> >>> >>> >>>> On 5 Apr 2007, at 12:49, James Page wrote: >>>> >>>> >>>> >>>> >>>>>> It feels to me like there may be a relatively simple "production >>>>>> only" bug happening >>>>>> >>>>>> >>>>>> >>>>> Can you confirm if this only happening in Tomcat and not in Jetty? >>>>> Are you testing production in Tomcat. >>>>> >>>>> DT optimizations are writing to 'out' less times. I am wondering if >>>>> the Writer is blocking. The reason I feel this is you say that you >>>>> are getting the performance hit at different times, and different >>>>> places. 2 seconds is a very long time compared .6 seconds even if the >>>>> code is not optimized. >>>>> >>>>> >>>>> >>>>> >>>> No, 2 requests per second, rather than 0.6/s but the same applies I >>>> imagine... >>>> >>>> Yes I think it might be blocking. It happens in production with Tomcat >>>> AND with Jetty locally. This is with concurrency tests, with 5 >>>> simultaneous clients and 1000 requests. >>>> >>>> >>>> >>>> >>>>> The quick thing to check also is that you are running java with the >>>>> server option -- just to check that it is not garbage collection, >>>>> getting in the way of your tests. Or even setting the gc collection >>>>> off for the test. >>>>> >>>>> >>>>> >>>>> >>>> It's concurrency/load I'm sure. It is easily seen on the server. I run >>>> "top", request a single form page and CPU rockets to 100% for a few >>>> seconds before the page comes back - every time. That would concur >>>> with our observed (repeatedly) < 1 req/s for forms, but 22/s for >>>> normal content. >>>> >>>> >>>> >>>> >>>>> Could you send me the headers that your page is generating, or tell >>>>> me if the server is returning the CONTENT-LENGTH and the CHUNKED >>>>> header.. >>>>> >>>>> >>>>> >>>>> >>>> I'll try to do this later. I've got to find some performance >>>> workarounds today and get them into the site or I'm going to get it in >>>> the neck. >>>> >>>> >>>> >>>> >>>>> The reason for looking at these headers is to see if the Server is >>>>> Writing out at the same time that you are Writing to the writer. In >>>>> the past I have seen many issues with Threads and Output streams. >>>>> >>>>> >>>>> >>>>> >>>> OK, sounds interesting. I'm not ruling this out, but Grails does write >>>> everything out to a String/char array in memory first: >>>> >>>> static GSPResonseWriter getInstance(ServletResponse response, int >>>> max) { >>>> return new GSPResonseWriter(response, new >>>> CharArrayWriter(max), max); >>>> } >>>> >>>> [not my typo!] >>>> >>>> I'm playing with changing max currently. I notice the flush() is >>>> synchronized so even though there should be no contention for the >>>> lock, it will be acquired and flushed 3 times per request currently >>>> for me with the default buffer size and the page in question. >>>> >>>> >>>> >>>> >>>>> If it is the writer blocking there is code I can look at that I wrote >>>>> in the past that should get around this problem. >>>>> >>>>> I was going to write this later for our site, but if this is causing >>>>> issues now. I will write an implementation now, and post it to the >>>>> mailing list. >>>>> >>>>> >>>>> >>>>> >>>> Thanks. I'm not so sure it is this. There are issues related to >>>> out.print, though it should be just writing to an in memory array. >>>> >>>> I am talking about 0.4.2 code by the way. Graeme has refactored GSP >>>> impl quite a bit in 0.5 but there are probably still things that can >>>> be done there. >>>> >>>> In 0.4.2 we do have several layers of buffering happening though. Each >>>> tag output is buffered to a StringWriter (with no default size set - >>>> this could be an issue for my large selects). The entire output of any >>>> GSP is dumped to a CharArrayWriter. Then it is flushed to the >>>> response's writer. THEN we have SiteMesh kicking in... >>>> >>>> Marc >>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe from this list please visit: >>>> >>>> http://xircles.codehaus.org/manage_email >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>> >>> ____________________________________________________________________________________ >>> Sucker-punch spam with award-winning protection. >>> Try the free Yahoo! Mail Beta. >>> http://advision.webevents.yahoo.com/mailbeta/features_spam.html >>> >>> --------------------------------------------------------------------- >>> To unsubscribe from this list please visit: >>> >>> http://xircles.codehaus.org/manage_email >>> >>> >>> >>> >>> >>> >>> >> >> >> ____________________________________________________________________________________ >> We won't tell. Get more on shows you hate to love >> (and love to hate): Yahoo! TV's Guilty Pleasures list. >> http://tv.yahoo.com/collections/265 >> >> --------------------------------------------------------------------- >> To unsubscribe from this list please visit: >> >> http://xircles.codehaus.org/manage_email >> >> >> >> >> >> > > > > > ____________________________________________________________________________________ > Don't pick lemons. > See all the new 2007 cars at Yahoo! Autos. > http://autos.yahoo.com/new_cars.html > > --------------------------------------------------------------------- > To unsubscribe from this list please visit: > > http://xircles.codehaus.org/manage_email > > > > > ____________________________________________________________________________________ Don't get soaked. Take a quick peek at the forecast with the Yahoo! Search weather shortcut. http://tools.search.yahoo.com/shortcuts/#loc_weather --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
Re: 0.4 performance trouble
|
In reply to this post by Marc Palmer Local
Has anyone tried to run TomCat with Grails inside a Java Profiler ? Surely this will show the most critical path of the application....
Barry On 4/5/07,
Marc Palmer <[hidden email]> wrote:
|
Re: 0.4 performance trouble
|
|
On 5 Apr 2007, at 15:17, Barry Lagerweij wrote: > Has anyone tried to run TomCat with Grails inside a Java Profiler ? > Surely this will show the most critical path of the application.... > Well, try it and see. Profiling is not that helpful as it gives you a LOT of data, and often there's not one particular thing that is taking a large % of the time, its lots and lots of little things that add up. For example in our previous profiling of Grails startup time we found that something like 50% of the time seemed to be spent in metaclass handling of GStrings. This may well be a bottleneck (at grails startup) but is not something that we can immediately do anything about anyway. Marc --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
Re: 0.4 performance trouble
|
In reply to this post by Barry Lagerweij
On 4/5/07, Barry Lagerweij <[hidden email]> wrote:
> Has anyone tried to run TomCat with Grails inside a Java Profiler ? Surely > this will show the most critical path of the application.... my coworker tried. It really didn't help narrow things down. Too much general information not enough specific. -- - kate = masukomi http://weblog.masukomi.org/ --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
In reply to this post by Marc Palmer Local
On 4/5/07, Marc Palmer <[hidden email]> wrote:
Not contention of the writers then..... I also tried caching the per-tag invocation StringWriter for the I know from J2me (which is _very much_ not J2SE) Exceptions are very expensive.
This is a dirty hack Strings as a whole are very expensive as they not mutable and any concentration need a StringBuffer in the background and then the creation of a new string. For the latter, I don't believe it is inherently slow to use EMC, but |
Re: 0.4 performance trouble
|
|
In reply to this post by Marc Palmer Local
FYI I just got around another 10% with my "improved" codecs plugin. I've attached the code to an issue. http://jira.codehaus.org/browse/GRAILS-1025 Marc --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
In reply to this post by Marc Palmer Local
IMHO You will find out allot more by bracketing the code with
Date time = new Date() //your code println object + "timetaken :" + (new Date()) -time then you can grep on the obj
On 4/5/07, Marc Palmer <[hidden email]> wrote:
|
|
|
In reply to this post by Marc Palmer Local
Yes, under the hood it uses CharArrayWriter.write(String) uses
System.arrayCopy() by calling String.getChars(). Which actually is what I think we should be calling ourselves anyway. But changing that won't have a big effect, I think. Yeah, now that you mention it I can see that the memory allocation wouldn't take too long (10 times re-allocating to copy a max of 8kb) and would really have a worse hit on minimal-output pages and tags (allocating extra memory). So maybe we can't pool the Writer objects. And maybe what I just suggested in that maybe we shouldn't be doing string copies at all when it's a large-enough string would also have too much overhead. Ok, I'll look at this again tonight. - Daiji Marc Palmer wrote: > > On 5 Apr 2007, at 14:50, D T wrote: > >> There's also a question about how we instantiate CharArrayWriter from >> GSPResonseWriter using the default constructor, which is: >> >> public CharArrayWriter() { >> this(32); >> } >> >> And it resizes by a factor of 2 every time it needs to do so. >> >> As for StringWriter (for tag invocations), it's even smaller: >> >> public StringWriter() { >> buf = new StringBuffer(); >> lock = buf; >> } >> >> public StringBuffer() { >> super(16); >> } > > > I'm a bit more concerned by the copying of the entire response String > to a char array with toCharArray() so it can be written to the > CharArrayWriter. Surely this is a new copy of the string as an > array... in my case 26KB, which equals 26000 loop iterations, albeit > with a JNI'd System.arraycopy I ... hope... > > Marc > > > --------------------------------------------------------------------- > To unsubscribe from this list please visit: > > http://xircles.codehaus.org/manage_email > > > > ____________________________________________________________________________________ Don't get soaked. Take a quick peek at the forecast with the Yahoo! Search weather shortcut. http://tools.search.yahoo.com/shortcuts/#loc_weather --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
In reply to this post by Marc Palmer Local
Better yet, use System.currentTimeMillis() to get closer to the native code.
James Page wrote: > IMHO You will find out allot more by bracketing the code with > > Date time = new Date() > //your code > println object + "timetaken :" + (new Date()) -time > > then you can grep on the obj > > On 4/5/07, *Marc Palmer* <[hidden email] > <mailto:[hidden email]>> wrote: > > > On 5 Apr 2007, at 15:17, Barry Lagerweij wrote: > > > Has anyone tried to run TomCat with Grails inside a Java Profiler ? > > Surely this will show the most critical path of the application.... > > > Well, try it and see. Profiling is not that helpful as it gives you a > LOT of data, and often there's not one particular thing that is > taking a large % of the time, its lots and lots of little things that > add up. > > For example in our previous profiling of Grails startup time we found > that something like 50% of the time seemed to be spent in metaclass > handling of GStrings. This may well be a bottleneck (at grails > startup) but is not something that we can immediately do anything > about anyway. > > Marc > > > > --------------------------------------------------------------------- > To unsubscribe from this list please visit: > > http://xircles.codehaus.org/manage_email > <http://xircles.codehaus.org/manage_email> > > ____________________________________________________________________________________ Finding fabulous fares is fun. Let Yahoo! FareChase search your favorite travel sites to find flight and hotel bargains. http://farechase.yahoo.com/promo-generic-14795097 --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
Re: 0.4 performance trouble
|
|
In reply to this post by Marc Palmer Local
On 5 Apr 2007, at 15:29, Marc Palmer wrote: > > FYI I just got around another 10% with my "improved" codecs plugin. > I've attached the code to an issue. > > http://jira.codehaus.org/browse/GRAILS-1025 Actually depending on the page - i.e. how many other tags there are (more big slower output tags make this difference less) - I am seeing huge performance increases. i.e. our home page runs circa 40req/s locally, previously around 22/s! :) Marc --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
«
Return to Grails - dev
|
1 view|%1 views
Loading...
| Powered by Nabble | Edit this page |