WSO2 Private PaaS on AWS EC2 doesn't complete the installation

Question

I'm trying to creanting an environment for WSO2 Private PaaS in Asia Pacific Region(ap-southeast-1) using Private PaaS AMI provided by WSO2 (ami-4e062c1c). I'm using Quick Start Guide and Quick Start Guide with Screencasts as references.

After go into private-paas directory I run boot.sh script, informing all information prompted and the installation starts. The log look like this:

root@ip-172-31-24-237:~/private-paas# ./boot.sh
Restoring from the Original template file /home/ubuntu/private-paas/stratos-installer/conf/setup.conf
Please enter a prefered domain name for the WSO2 Private PaaS environment : paas.wso2.com.br
Using Stratos domain: paas.wso2.com.br
Rather than invoking init scripts through /etc/init.d, use the service(8)
utility, e.g. service hostname restart

Since the script you are attempting to invoke has been converted to an
Upstart job, you may also use the stop(8) and then start(8) utilities,
e.g. stop hostname ; start hostname. The restart(8) utility is also available.
hostname stop/waiting
Puppet master IP address is 172.31.24.237
Puppet master hostname is puppet.paas.wso2.com.br

Skipping MySQL installation...

Deploying Puppet files to /etc/puppet. Please wait...
Puppet files successfully deployed.
Configuring Puppet scripts...
Restarting Puppet master
 * Restarting puppet master                                                                             [ OK ]
Puppet scripts configuration completed.
Do you need to deploy AS (Application Server) service? [y/n] y
Do you need to deploy BPS (Business Process Server) service? [y/n] n
Do you need to deploy ESB (Enterprise Service Bus) service? [y/n] y
Do you need to deploy APIM (API Manager) service? [y/n] y
Creating registry database: registry
Restoring from the Original template file /home/ubuntu/private-paas/resources/dbscripts/registry.sql
Creating config database: sm_config
Restoring from the Original template file /home/ubuntu/private-paas/resources/dbscripts/config.sql
You selected Amazon EC2.
Are you in a EC2 VPC Environment? [y/n] : y
Enter EC2 identity : AKIAIUIM2EV7KCT76VRQ
Enter EC2 credentials :
Enter EC2 owner id : 960661076368
Enter EC2 keypair name : paas
Enter EC2 availability zone : ap-southeast-1
Enter EC2 security group IDs : sg-a31586c6
Enter EC2 VPC Subnet ID : subnet-4e369739
   Below are the available regions in Amazon EC2
   ap-northeast-1 - Asia Pacific (Tokyo) Region
   ap-southeast-1 - Asia Pacific (Singapore) Region
   ap-southeast-2 - Asia Pacific (Sydney) Region
   eu-west-1 - EU (Ireland) Region
   sa-east-1 - South America (Sao Paulo) Region
   us-east-1 - US East (Northern Virginia) Region
   us-west-1 - US West (Northern California) Region
   us-west-2 - US West (Oregon) Region
Enter the region of the IaaS you want to spin up instances : ap-southeast-1
You have selected the profile : default
user provided in conf/setup.conf is ubuntu.
In default profile CEP will be configured.
Extracting Apache Stratos
Extracting ActiveMQ
~/private-paas/install/apache-stratos-default ~/private-paas
In repository/conf/carbon.xml
In repository/conf/jndi.properties
~/private-paas
Configuring the Cloud Controller
In repository/conf/cloud-controller.xml
~/private-paas/install/apache-stratos-default ~/private-paas
~/private-paas
~/private-paas/install/apache-stratos-default ~/private-paas
~/private-paas
End configuring the Cloud Controller
Configuring the Autoscaler
~/private-paas/install/apache-stratos-default ~/private-paas
In repository/conf/autoscaler.xml
~/private-paas
End configuring the Autoscaler
Configuring Stratos Manager
~/private-paas/install/apache-stratos-default ~/private-paas
In repository/conf/cartridge-config.properties
In repository/conf/datasources/master-datasources.xml
In repository/conf/registry.xml
~/private-paas
Creating userstore database
~/private-paas/stratos-installer/resources ~/private-paas
~/private-paas
End configuring the SM
Configuring the Complex Event Processor
~/private-paas/install/apache-stratos-default ~/private-paas
In outputeventadaptors
End configuring the Complex Event Processor
~/private-paas
Changing owner of /home/ubuntu/private-paas/install to ubuntu:ubuntu
Apache Stratos configuration completed successfully
Apache Stratos Installer completed successfully!
Restoring from the Original template file /home/ubuntu/private-paas/resources/json/ec2/lb-cart.json
Configuring Application Server...
Creating config database: as_config
Restoring from the Original template file /home/ubuntu/private-paas/resources/dbscripts/config.sql
Restoring from the Original template file /home/ubuntu/private-paas/resources/json/ec2/appserver-cart.json
Restoring from the Original template file /home/ubuntu/private-paas/resources/json/ec2/appserver-cart-mgt.json
Restoring from the Original template file /home/ubuntu/private-paas/resources/json/ec2/appserver-cart-worker.json
Configuring Enterprise Service Bus...
Creating a back-up of the file /etc/puppet/modules/esb/manifests/params.pp
Creating config database: esb_config
Restoring from the Original template file /home/ubuntu/private-paas/resources/dbscripts/config.sql
Restoring from the Original template file /home/ubuntu/private-paas/resources/json/ec2/esb-cart.json
Restoring from the Original template file /home/ubuntu/private-paas/resources/json/ec2/esb-cart-mgt.json
Restoring from the Original template file /home/ubuntu/private-paas/resources/json/ec2/esb-cart-worker.json
Creating config database: is_config
Restoring from the Original template file /home/ubuntu/private-paas/resources/dbscripts/config.sql
Creating APIM database: apim_db
Creating a back-up of the file /home/ubuntu/private-paas/resources/dbscripts/apim.sql
Creating config database: apim_store_config
Restoring from the Original template file /home/ubuntu/private-paas/resources/dbscripts/config.sql
Creating config database: apim_gateway_config
Restoring from the Original template file /home/ubuntu/private-paas/resources/dbscripts/config.sql
Creating config database: apim_keymanager_config
Restoring from the Original template file /home/ubuntu/private-paas/resources/dbscripts/config.sql
Creating a back-up of the file /home/ubuntu/private-paas/resources/json/ec2/gateway.json
Creating a back-up of the file /home/ubuntu/private-paas/resources/json/ec2/gatewaymgt.json
Creating a back-up of the file /home/ubuntu/private-paas/resources/json/ec2/keymanager.json
Creating a back-up of the file /home/ubuntu/private-paas/resources/json/ec2/publisher.json
Creating a back-up of the file /home/ubuntu/private-paas/resources/json/ec2/store.json
WSO2 Private PaaS product configuration completed.
Starting BAM core service...

However instalation process stop in this part:

nohup: ignoring input and redirecting stderr to stdout
The authenticity of host 'localhost (127.0.0.1)' can't be established.
ECDSA key fingerprint is 70:4e:5d:3e:f3:70:fa:5f:48:11:e9:bb:df:69:db:f0.
Are you sure you want to continue connecting (yes/no)? yes

After ~= 3 hours I closed the terminal, it makes no sense to take so much time. I searched for logs at:

• /home/ubuntu/private-paas/install/apache-stratos-default/repository/logs/: no logs;
• /home/ubuntu/private-paas/install/wso2is-5.0.0/repository/logs/: no logs;
• /home/ubuntu/private-paas/install/wso2bam-2.4.0/repository/logs/: wso2carbon.log has this error:

TID: [0] [BAM] [2015-08-20 12:56:13,247] ERROR {org.wso2.carbon.webapp.mgt.AbstractWebappDeployer} -  Error occurred while deploying webapp : /home/ubuntu/private-paas/install/wso2bam-2.4.0/repository/deployment/server/webapps/datareceiver.war {org.wso2.carbon.webapp.mgt.AbstractWebappDeployer}
org.wso2.carbon.CarbonException: Error while deploying webapp: StandardContext[datareceiver.war].File[/home/ubuntu/private-paas/install/wso2bam-2.4.0/repository/deployment/server/webapps/datareceiver.war]
    at org.wso2.carbon.webapp.mgt.TomcatGenericWebappsDeployer.handleWebappDeployment(TomcatGenericWebappsDeployer.java:283)
    at org.wso2.carbon.webapp.mgt.TomcatGenericWebappsDeployer.handleWarWebappDeployment(TomcatGenericWebappsDeployer.java:174)
    at org.wso2.carbon.webapp.mgt.TomcatGenericWebappsDeployer.handleHotDeployment(TomcatGenericWebappsDeployer.java:141)
    at org.wso2.carbon.webapp.mgt.TomcatGenericWebappsDeployer.deploy(TomcatGenericWebappsDeployer.java:116)
    at org.wso2.carbon.webapp.mgt.AbstractWebappDeployer.deployThisWebApp(AbstractWebappDeployer.java:140)
    at org.wso2.carbon.webapp.mgt.AbstractWebappDeployer.deploy(AbstractWebappDeployer.java:90)
    at org.wso2.carbon.webapp.deployer.WebappDeployer.deploy(WebappDeployer.java:42)
    at org.apache.axis2.deployment.repository.util.DeploymentFileData.deploy(DeploymentFileData.java:136)
    at org.apache.axis2.deployment.DeploymentEngine.doDeploy(DeploymentEngine.java:810)
    at org.apache.axis2.deployment.repository.util.WSInfoList.update(WSInfoList.java:144)
    at org.apache.axis2.deployment.RepositoryListener.update(RepositoryListener.java:377)
    at org.apache.axis2.deployment.RepositoryListener.checkServices(RepositoryListener.java:254)
    at org.apache.axis2.deployment.DeploymentEngine.loadServices(DeploymentEngine.java:139)
    at org.wso2.carbon.core.CarbonAxisConfigurator.loadServices(CarbonAxisConfigurator.java:464)
    at org.apache.axis2.context.ConfigurationContextFactory.createConfigurationContext(ConfigurationContextFactory.java:95)
    at org.wso2.carbon.core.CarbonConfigurationContextFactory.createNewConfigurationContext(CarbonConfigurationContextFactory.java:65)
    at org.wso2.carbon.core.init.CarbonServerManager.initializeCarbon(CarbonServerManager.java:398)
    at org.wso2.carbon.core.init.CarbonServerManager.start(CarbonServerManager.java:219)
    at org.wso2.carbon.core.internal.CarbonCoreServiceComponent.activate(CarbonCoreServiceComponent.java:77)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
    at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
    at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:347)
    at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
    at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
    at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
    at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
    at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
    at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
    at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
    at org.eclipse.equinox.http.servlet.internal.Activator.registerHttpService(Activator.java:81)
    at org.eclipse.equinox.http.servlet.internal.Activator.addProxyServlet(Activator.java:60)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.init(ProxyServlet.java:40)
    at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.init(DelegationServlet.java:38)
    at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1267)
    at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1186)
    at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1081)
    at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5027)
    at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5314)
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
    at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1559)
    at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1549)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class sun.awt.SunToolkit
    at sun.awt.AppContext$2.run(AppContext.java:271)
    at sun.awt.AppContext$2.run(AppContext.java:260)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.awt.AppContext.initMainAppContext(AppContext.java:260)
    at sun.awt.AppContext.access$200(AppContext.java:133)
    at sun.awt.AppContext$3.run(AppContext.java:314)
    at sun.awt.AppContext$3.run(AppContext.java:298)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.awt.AppContext.getAppContext(AppContext.java:297)
    at sun.awt.AppContext$6.getContext(AppContext.java:841)
    at sun.misc.SharedSecrets.getJavaAWTAccess(SharedSecrets.java:200)
    at java.util.logging.LogManager.getUserContext(LogManager.java:372)
    at java.util.logging.LogManager.getLogger(LogManager.java:1029)
    at java.util.logging.LogManager.demandLogger(LogManager.java:424)
    at java.util.logging.Logger.demandLogger(Logger.java:343)
    at java.util.logging.Logger.getLogger(Logger.java:390)
    at org.apache.juli.logging.DirectJDKLog.<init>(DirectJDKLog.java:71)
    at org.apache.juli.logging.DirectJDKLog.getInstance(DirectJDKLog.java:196)
    at org.apache.juli.logging.LogFactory.getInstance(LogFactory.java:170)
    at org.apache.juli.logging.LogFactory.getInstance(LogFactory.java:242)
    at org.apache.juli.logging.LogFactory.getLog(LogFactory.java:293)
    at org.apache.catalina.session.ManagerBase.<init>(ManagerBase.java:64)
    at org.apache.catalina.session.StandardManager.<init>(StandardManager.java:63)
    at org.wso2.carbon.webapp.mgt.CarbonTomcatSessionManager.<init>(CarbonTomcatSessionManager.java:48)
    at org.wso2.carbon.webapp.mgt.TomcatGenericWebappsDeployer.handleWebappDeployment(TomcatGenericWebappsDeployer.java:247)
    ... 54 more

Although this error, I think this isn't the cause to no complete installation, right? When I check for java processes, I can see only processes for hadoop and WSO2 BAM, nothing for stratos or other WSO2 products. So WSO2 products isn't installed.

For a try I create a private AMI from AMI provided by WSO2 and moved for another region. In this case I can run at least WSO2 Private PaaS Console (stratos), but the instances for APIM, AS, etc. isn't created due the wrong AMI base ID in the scripts.

Finally, the questions:

• does anyone have any idea what not complete the installation?
• does anyone have an alternative to provided this environment?
• there is a more recent WSO2 Private PaaS AMI?
• there is some documentation beyond that I am using?

Answer 1

The solution makes no sense to me, due to the indirect relation between cause and effect, but after some mails exchange with WSO2 support and 3 days after these emails, it was suggested to disable BAM installation.

This can be done leaving empty bam_enabled property in conf.sh file, present at private-paas directory. Will look like this:

export bam_enabled=""

Apparently puppet script provided in the AMI to deploy WSO2 Private PaaS isn't stable, since I could deploy WSO2 Private PaaS in another AWS region installing WSO2 BAM.

Now I go find a better way to provide WSO2 Private PaaS, EC2 region independent, making my own deploy script or improving the one available in the repository.

Answer 2

About your first question, its more for Linux related than PaaS, its a problem with server SSH keys and with your hostname you must review the SSH environment before start the installation and put the correct hostname in your machine and test the names.

Try for example:

ssh -l localuser localhost

What do you get?

Then try:

ssh -l localuser paas.wso2.com.br

What do you get?