Default heap size and OOM
Locked
 
|
Hi,
We use the OpenShift cartridge to deploy AeroGear UnifiedPush server version 1.0.3. We deploy it on a medium gear. Am I expected to increase the maximum heap size for JVM, set by default to be 512m? We are getting OutOfMemoryError frequently. And it's often very easily reproducible by sending 10 or so consecutive messages from AeroGear's admin console. I can provide more details, but I wanted to see if I was supposed to increase the heap size. Thank you. |
Re: [Aerogear-users] Default heap size and OOM
|
|
Hi,
Farah and I are looking to remove the JGroups subsystem. This has been a boost on other cartridges (that do not need clustering) as well. Will keep you updated On Wed, Jun 17, 2015 at 7:14 PM, mo <[hidden email]> wrote: Hi, Matthias Wessendorf
blog: http://matthiaswessendorf.wordpress.com/ sessions: http://www.slideshare.net/mwessendorf twitter: http://twitter.com/mwessendorf _______________________________________________ Aerogear-users mailing list [hidden email] https://lists.jboss.org/mailman/listinfo/aerogear-users |
|
|
Thanks, Matthias. I take your reply to mean "No, one shouldn't have to increase JVM heap size."
Let me list some more parameters, in the hope that you might spot something obvious that we can correct on our side. It's been difficult to troubleshoot this, because when the problem happens we can't even SSH into the server. I believe whatever the problem is, it's exhausting resources on the server. Parameters: 1. Deployed on OpenShift using the cartridge found here: https://github.com/aerogear/openshift-origin-cartridge-aerogear-push 2. AeroGear UnifiedPush Server 1.0.3 and WildFly, MySQL 5.5, Medium gear. 3. One application is defined with one variant (simple-push), 131 installations registered. 4. We are using all default settings. We haven't customized the JVM max heap size, for example. The server log tells me "-Xmx512m". Here are the reproduction steps. This happened reliably two days ago. We couldn't reproduce this yesterday after a fresh restart of AeroGear. 1. Logged into AeroGear as admin 2. Select "Send Push" 3. Select an application 4. Hit "Send Push Notification", multiple times one after another. About 10 consecutive messages would cause OutOfMemoryError on our server. Here's a relevant snippet of the server log: [0m[0m17:03:29,511 INFO [PushNotificationSenderEndpoint] (default task-28) Message submitted to PushNetworks for further processing [0m[0m17:03:29,513 INFO [JPAPushMessageInformationDao] (EJB default - 6) Processing send request with '[alert=, criteria=[aliases=null, deviceTypes=null, categories=null, variants=null], time-to-live=-1]' payload [0m[31m17:03:29,837 ERROR [org.jboss.as.ejb3.invocation] (EJB default - 6) JBAS014134: EJB Invocation failed on component SenderServiceImpl for method public abstract void org.jboss.aerogear.unifiedpush.message.SenderService.send(org.jboss.aerogear.unifiedpush.api.PushApplication,org.jboss.aerogear.unifiedpush.message.UnifiedPushMessage): javax.ejb.EJBException: JBAS014580: Unexpected Error at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleExceptionInOurTx(CMTTxInterceptor.java:187) [wildfly-ejb3-8.1.0.Final.jar:8.1.0.Final] at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:275) [wildfly-ejb3-8.1.0.Final.jar:8.1.0.Final] . . . [wildfly-ejb3-8.1.0.Final.jar:8.1.0.Final] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_79] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_79] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_79] at org.jboss.threads.JBossThread.run(JBossThread.java:122) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) [rt.jar:1.7.0_79] at java.lang.Thread.start(Thread.java:714) [rt.jar:1.7.0_79] at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1056) [jsse.jar:1.7.0_79] at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1332) [jsse.jar:1.7.0_79] at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1359) [jsse.jar:1.7.0_79] . . . org.jboss.aerogear.unifiedpush.message.sender.SimplePushNotificationSender.put(SimplePushNotificationSender.java:111) [unifiedpush-push-1.0.3.jar:1.0.3] at org.jboss.aerogear.unifiedpush.message.sender.SimplePushNotificationSender.sendPushMessage(SimplePushNotificationSender.java:65) [unifiedpush-push-1.0.3.jar:1.0.3] at org.jboss.aerogear.unifiedpush.message.SenderServiceImpl.send(SenderServiceImpl.java:110) [unifiedpush-push-1.0.3.jar:1.0.3] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_79] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_79] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_79] at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_79] at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) . . . at org.jboss.as.ejb3.component.interceptors.NonPooledEJBComponentInstanceAssociatingInterceptor.processInvocation(NonPooledEJBComponentInstanceAssociatingInterceptor.java:59) [wildfly-ejb3-8.1.0.Final.jar:8.1.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:273) [wildfly-ejb3-8.1.0.Final.jar:8.1.0.Final] ... 38 more |
Re: [Aerogear-users] Default heap size and OOM
|
|
Hi ,
On Thu, Jun 18, 2015 at 4:57 PM, mo <[hidden email]> wrote: Thanks, Matthias. I take your reply to mean "No, one shouldn't have to yeah, if you really want to, you can try tweaking the jvm heap size (as long as you still use a function of the max memory available on the gear)
But also please review the changes done in You could port them over to your instance. Makes sense ? More coming soon.
Matthias Wessendorf
blog: http://matthiaswessendorf.wordpress.com/ sessions: http://www.slideshare.net/mwessendorf twitter: http://twitter.com/mwessendorf _______________________________________________ Aerogear-users mailing list [hidden email] https://lists.jboss.org/mailman/listinfo/aerogear-users |
Re: [Aerogear-users] Default heap size and OOM
|
|
Hi Mo, the PR has been merged, and it should improve the situation a bit PS: next will be removing SimplePush from the cartridge On Thu, Jun 18, 2015 at 5:17 PM, Matthias Wessendorf <[hidden email]> wrote:
Matthias Wessendorf
blog: http://matthiaswessendorf.wordpress.com/ sessions: http://www.slideshare.net/mwessendorf twitter: http://twitter.com/mwessendorf _______________________________________________ Aerogear-users mailing list [hidden email] https://lists.jboss.org/mailman/listinfo/aerogear-users |
|
|
Awesome. Thank you, Matthias.
I'm currently revising our client side code, as it is likely contributing to wasted resources on the server side. I will post a report after the revision and testing. |
«
Return to aerogear-users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |