Jenkins EC2 plugin: instances not starting

Question

This is my Jenkins EC2 configuration:

• URL: $JENKINS_URL/configureClouds/
• Add new cloud: Amazon EC2
• Name: Amazon EC2 eu-central-1
• Amazon EC2 Credentials: AKIA...
• Region: eu-central-1
• EC2 Key Pair's Private Key: ubuntu
• Test connection: success
• Advanced...
• Instance Cap: 3
• No delay provisioning: checked
• Add AMI
• Description: Linux node
• AMI ID: ami-0293...
• Check AMI: 05052029...
• Instance Type: T3aMedium
• EBS Optimized: checked
• Monitoring: checked
• T2 Unlimited: checked
• Security group names: sg-0c2d... (opens SSH port 22)
• Remote FS root: ./jenkins
• Remote user: ubuntu
• AMI Type: unix
• Labels: aws ubuntu linux
• Usage: Use this node as much as possible
• Idle termination time: 30
• Advanced...
• Number of executors: 2
• Stop/Disconnect on Idle Timeout: checked
• Minimum number of instances: 1
• Minimum number of spare instances: 0
• Instance cap: 10
• Block device mapping: /dev/sda1=snap-0eadbe3f...:200:true:gp2, /dev/sdb=ephemeral0, /dev/sdc=ephemeral1
• Associate Public IP: checked
• Connection Strategy: Public DNS
• Host Key Verification Strategy: off
• Maximum Total Uses: 10
• Environment variables: checked (not listing all environment variables)
• Tool locations: checked (not listing all tool locations)

With this configuration, I would expect that at least 1 EC2 instance would be started, but no instance is started.

In the nodes page in Jenkins when I hit the provision via button, I get an error:

Oops! A problem occurred while processing the request. Logging ID=8ead3651-3809-4a47-984c-e0e494c705bb

In /log/all I have:

Apr 14, 2021 5:34:37 PM INFO hudson.plugins.ec2.SlaveTemplate getImage
Getting image for request {ExecutableUsers: [],Filters: [],ImageIds: [ami-0293c4ed***],Owners: []}
Apr 14, 2021 5:34:37 PM INFO hudson.plugins.ec2.SlaveTemplate logProvisionInfo
SlaveTemplate{description='Linux node', labels='aws ubuntu linux'}. Considering launching
Apr 14, 2021 5:34:37 PM INFO hudson.plugins.ec2.SlaveTemplate setupRootDevice
AMI had /dev/sda1
Apr 14, 2021 5:34:37 PM INFO hudson.plugins.ec2.SlaveTemplate setupRootDevice
{DeleteOnTermination: true,SnapshotId: snap-0eadbe3f***,VolumeSize: 20,VolumeType: gp2,Encrypted: false}
Apr 14, 2021 5:34:37 PM INFO hudson.plugins.ec2.SlaveTemplate logProvisionInfo
SlaveTemplate{description='Linux node', labels='aws ubuntu linux'}. Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Stop
Apr 14, 2021 5:34:37 PM INFO hudson.plugins.ec2.SlaveTemplate logProvisionInfo
SlaveTemplate{description='Linux node', labels='aws ubuntu linux'}. Looking for existing instances with describe-instance: {Filters: [{Name: image-id,Values: [ami-0293c4ed***]}, {Name: instance-type,Values: [t3a.medium]}, {Name: key-name,Values: [***]}, {Name: tag:jenkins_server_url,Values: [https://jenkins.***.com/]}, {Name: tag:jenkins_slave_type,Values: [demand_Linux node]}],InstanceIds: [],}
Apr 14, 2021 5:34:37 PM WARNING hudson.init.impl.InstallUncaughtExceptionHandler handleException
Caught unhandled exception with ID c080ae42-6b7b-47aa-93ea-1b8064503c1c
com.amazonaws.services.ec2.model.AmazonEC2Exception: Value () for parameter groupId is invalid. The value cannot be empty (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue; Request ID: da74dbbf-0685-45ac-8454-c3f5d1b4c700; Proxy: null)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1819)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1403)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1372)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
    at com.amazonaws.services.ec2.AmazonEC2Client.doInvoke(AmazonEC2Client.java:29240)
    at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:29207)
    at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:29196)
    at com.amazonaws.services.ec2.AmazonEC2Client.executeRunInstances(AmazonEC2Client.java:28011)
    at com.amazonaws.services.ec2.AmazonEC2Client.runInstances(AmazonEC2Client.java:27980)
    at hudson.plugins.ec2.SlaveTemplate.provisionOndemand(SlaveTemplate.java:1100)
    at hudson.plugins.ec2.SlaveTemplate.provisionOndemand(SlaveTemplate.java:1042)
    at hudson.plugins.ec2.SlaveTemplate.provision(SlaveTemplate.java:867)
    at hudson.plugins.ec2.EC2Cloud.getNewOrExistingAvailableSlave(EC2Cloud.java:693)
    at hudson.plugins.ec2.EC2Cloud.doProvision(EC2Cloud.java:430)
    at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)
    at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:396)
    at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:408)
    at org.kohsuke.stapler.interceptor.RequirePOST$Processor.invoke(RequirePOST.java:77)
    at org.kohsuke.stapler.PreInvokeInterceptedFunction.invoke(PreInvokeInterceptedFunction.java:26)
    at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:212)
    at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:145)
    at org.kohsuke.stapler.MetaClass$11.doDispatch(MetaClass.java:536)
    at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898)
    at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:281)
    at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:694)
    at org.kohsuke.stapler.Stapler.service(Stapler.java:240)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791)
    at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
    at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:248)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
    at jenkins.security.ResourceDomainFilter.doFilter(ResourceDomainFilter.java:76)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
    at jenkins.telemetry.impl.UserLanguages$AcceptLanguageFilter.doFilter(UserLanguages.java:129)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
    at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:111)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
    at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:60)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
    at com.cloudbees.jenkins.support.slowrequest.SlowRequestFilter.doFilter(SlowRequestFilter.java:37)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
    at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
    at org.jenkinsci.plugins.modernstatus.ModernStatusFilter.doFilter(ModernStatusFilter.java:50)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
    at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:64)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
    at jenkins.metrics.impl.MetricsFilter.doFilter(MetricsFilter.java:125)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
    at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
    at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:153)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:92)
    at jenkins.security.AcegiSecurityExceptionFilter.doFilter(AcegiSecurityExceptionFilter.java:52)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)
    at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)
    at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119)
    at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)
    at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)
    at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:101)
    at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:92)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)
    at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:218)
    at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:212)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)
    at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)
    at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)
    at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)
    at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:62)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)
    at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:109)
    at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:168)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
    at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:51)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
    at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
    at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
    at jenkins.security.SuspiciousRequestFilter.doFilter(SuspiciousRequestFilter.java:36)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:578)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1435)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1350)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
    at org.eclipse.jetty.server.Server.handle(Server.java:516)
    at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388)
    at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:279)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
    at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
    at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:383)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:882)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1036)
    at java.lang.Thread.run(Thread.java:748)

Answer 1

Write up of the comments for anyone else looking for help diagnosing EC2 Agent Plugin issue.

• When you have configured your agents go to the Nodes page (Jenkins URL/computer)
• Hit the button to Provision a new agent from your cloud
• If there is a configuration issue you will get Evil Jenkins and a Logging ID
• Go to Jenkins logs page (Jenkins URL/log/all) and search that ID
• This should give you the stack trace from the AWS SDK call which will help you to narrow down whether is it missing config or IAM permissions etc at fault

If there were no config errors you would get taken to the node that is being launched config page where you would be able to see its EC2 startup log to check for any User Data or AMI issues.

Answer 2

For this particular issue 'Value () for parameter groupId is invalid' the plugin is expecting you to provide a list or at least one subnet-id in configuration.

I faced the same issue and was resolved with this.