Three ways to create a micro-service on Google Cloud - Part 2: Compute Engine

This is part two of a three part blog post describing how I used three distinct Google Cloud technologies to run the same micro-service. I would read Part 1 before reading this.

Part 2: Compute Engine
Compute engine is Google's service for renting Virtual Machines. This is a lower-level service than Google App Engine. It is essentially like renting a computer that runs in the cloud. Except it's a virtual computer. But from the developer's perspective (or user's perspective) it's mostly indistinguishable from a physical computer. Google Compute Engine is roughly similar to Amazon's EC2.

A neat part of Compute Engine's is a related service called Launcher. It has a bunch of common configurations (software stacks) from which to create and launch your new VM. For example, I chose a package called Tomcat Stack (a stack with Tomcat and MySQL on Debian 7).

The general steps to deploy my micro-service were as follows:

1. Go into Google Cloud's Admin Console and create a new project (we already had a project setup for GAE called cspService, so we just used that one).
   
   
2. Goto the Launcher Tomcat Stack URL:
   
   https://cloud.google.com/launcher/solution/click-to-deploy-images/tomcat
   
   
3. Click on the "Launch on Google Cloud Platform" button:
   
   
   
   
4. Choose a project to deploy into:
   
   
   
   
   
   
5. Adjust settings and deploy.
   
   
   
   
   Note that I chose a CPU with 104 GB RAM. Our compile function requires a lot of memory for large input files.
   
   
6. Now we wait for about 3 to 5 minutes. After our new Compute Engine instance is up and running, we are provided with some useful information:
• The public ip address of our new new instance
• The instance name (for use in the admin console - ours was tomcat1-tomcat-hjsn)
• The zone (us-central1-f in our case)
• Instructions on how to ssh into the machine and copy files up to the machine.
  
  
7. Click on the public ip address to allow HTTP traffic to our new instance:
   
   
   
   
8. Finally, I copied the exploded web app from my laptop up to my new Compute Engine instance:
   
   gcloud compute copy-files
       target/cspService \
       tomcat1-tomcat-hjsn:/var/lib/tomcat7/webapps \
       --zone us-central1-f
   
   The command line tools come part of the Google Cloud SDK for Java.
   
   

And that's it. My server is up and running. Anyone can now surf to my WebApp.

So, compared to the GAE deployment, what did I gain and what did I lose? 

Gains 

• There is no limit to the amount of time our servlet is allowed to spend handling a single request.
• There is no limit on JVM heap size.
• We can now call servlet methods specific to the Servlet 3.0 API (the Tomcat Stack we chose came with tomcat 7, which supports the Servlet API 3.0).
• And in general, I now have a much greater level of flexibility.

Losses

• Bigger pain in the butt in the initial set up and deployment.
• No load balancer. If I want a load balancer I have to take extra steps to set this up.
• No multiple instances. If I want multiple instances to handle load, I need to create them manually and deploy them. And then setup the load balancer to deal with those instances.
• No more multiple concurrent versions

So here is what I would like:

• The convenience of the GAE solution.
• The super easy scalability and reliability of GAE solution. 
• The flexibility of the Compute Engine solution. 

Here are two points to help set the stage for part three of our series:

1. When talking about GAE's super easy scalability and reliability what we are really talking about are some magic technologies running behind the scenes. Things like:
• Clusters that run multiple replicas of your app
• Load balancers that sit in front of those replicas
• Some type of management software to ensure that the replicas are alive and spin up new ones as needed.
  
  
2. One of the hot technologies these days in the area of flexible yet convenient app deployment is Docker. 

In part three I will show you how we deployed this same app using Google Container Engine which combine the benefits of Docker with Google's open source clustering technology, kubernetes. 

Three ways to create a micro-service on Google Cloud - Part 1: App Engine

So I had this function, written in Java. And I wanted to turn it into a micro-service. The function is basically an AI compiler. It takes a set of business constraints and compiles them into a more tractable format. It takes one big string as input and returns one big string as output. The function is stateless and has no side-effects.  It is a perfect candidate for a micro-service.

I have a few reasons for wanting to do this:

1. Scalability and Performance. This function is compute-intensive. And it's a performance bottle neck for the application. I would like to scale this function independently from the rest of the application. The stateless nature of this function makes it a good candidate for replication and load balancing. 
2. Non-Java Clients. By turning the function into an HTTP service, non-java clients can easily access it.
3. Versioning. It's easy to deal with multiple versions of a service. But dealing with multiple jar files is less easy. I can deploy an updated version of a service without having to redeploy apps that consume the service. This is not so easy with jar files.

As an experiment I decided to create three distinct deployments of this micro-service, using three distinct Google Cloud offerings:

1. Google App Engine (GAE)
2. Google Compute Engine
3. Google Container Engine

The fact is, I could use any of these three technologies to create my micro-service. Each would work. But there are pros and cons to each. One of the three ended up working best for our needs. By sharing this experiment, I thought I could provide some useful insight for those facing similar choices.

Note: this is not meant to be a thorough tutorial of anyone of these technologies but rather to give you a feel for how the three offerings differ.

Part 1: Google App Engine

The simplest and easiest of the three is Google App Engine. GAE is the highest level of the three, requiring the least amount of system administration. You can sort of think of GAE as a Tomcat hosting service. Well not technically. We don't really know if they are using Tomcat. But they are using something similar (I have heard GAE is based on Jetty). All you really need to know is that App Engine is a Servlet 2.5 Web Container.

Here is what you need to do:

1. Create a new project (I called ours cspService) in Google Cloud's Admin Console.
2. Create a Java WebApp (based on the Servlet 2.5 spec).
3. Add a minimal GAE config file (WEB-INF/appengine-web.xml).
4. Deploy the exploded WebApp to GAE using Google's command line tool:
   
   appcfg.sh -A cspservice-1065 update target/cspService
   
   where:
   
     cspservice-1065 is my project id
     target/cspService is the path to my exploded WebApp on my local system.
   
   The command line tools come part of the Google App Engine SDK for Java.

Here is what you don't have to do:

• Own a server computer
• Rent a machine or a virtual machine (at least not explicitly)
• Know what Linux is or how to use it
• Know anything about VM's
• Create a ticket with your web ops group
• Most types of system admin work

Now that Google is hosting your WebApp you get:

• Google scaling
• Load balancing
• Multiple replicas of your app to handle load
• Google automatically scales the number of instances (replicas) based on usage patterns
• Google reliability
• Google system administration
• Google monitoring
• Multiple concurrent versions

To put things into perspective timewise: it took me about 20 minutes to wrap my Java function in a servlet and WebApp. After that, it took about 5 minutes to deploy the exploded WebApp to GAE. I now have a live running micro-service, ready to handle 1000s of concurrent users. All in under half-an-hour.

At first, everything was awesome. But after some further testing, we ran up against a few issues:

DeadlineExceededException

Some of our larger input files take a super long time to compile. Well it turns out that GAE, by default, limits the amount of time our servlet is allowed to spend handling any one request. By default, that limit is 60 seconds. This works for 90% of our use-cases. But that's not good enough. Fortunately, this limit can be circumvented easily on GAE. We have two options:

1. Switch from GAE's default setting of automatic-scaling to basic-scaling.  This is done in WEB-INF/appengine-web.xml. This has other consequences that I won't go into, but suffice it to say, it works for our app.
2. Our other option is to use a feature of GAE called Task Queues. These are meant for long running background jobs, so as to not slow down client requests. Well, in this micro-service, compiling is not a background job its the only job. So it doesn't seem appropriate to use this option. Another point to note is that Task Queues require the use of some Google-specific API's. So if WebApp portability is important to you, this may be a factor.

In our case, we decided that option #1 was the simplest way to go.

Servlet API 2.5
I had inadvertently taken advantage of a feature from Servlet API 3.0 in our servlet. Not every execution path uses this feature, so it didn't come up right away. But eventually we got a java.lang.NoSuchMethodError. The reason: GAE does not support Servlet 3.0 API. It supports 2.5.

The problem here is a classic one: my test environment was different than the production environment (stay tuned for part 3 of this post, which addresses this issue).

Anyway, this issue was easy to work around. We just modified the servlet to not use that Servlet API 3.0 specific method. It was not a deal breaker.

Memory Limits

Our constraint compiler uses a lot of memory. And the larger the input file, the more memory it uses. The maximum heap space allowed by GAE is 1024 MB. This works for 90% of our use-cases. But it's a deal breaker. Unlike the "long running" issue, there appears to be no simple work-around for the memory issue within GAE.

GAE Wrap Up

Before moving on to the other two Google Cloud offerings let me note that there are a few other restrictions beyond what I have already noted (for example, GAE only supports web apps written in Java, Python, PHP or Go) but for the most part, they are few. I believe that our high memory requirements are very unique. For many many applications, GAE is the easiest and smartest choice.

Google Compute Engine and Google Container Engine are much more flexible and powerful than GAE. But that comes with additional complexity and administration.

The other day, a client shared with me an internal debate they were having. They were trying to decide which of these three offerings to choose. This was a big shot managerial person who oversees dozens of projects. So this was not a project-specific decision she was trying to make. She was trying to come up with some sort of policy. She was leaning more toward Google Compute Engine and less toward GAE. 

I didn't think to say this at the time, but I think the correct advice is this: don't standardize on just one of these offerings. If you are overseeing a dozen projects, why not run some apps on GAE and others on Google Compute Engine or Google Container Engine. If your requirements fit within the limitations of GAE, than go with GAE. Otherwise go with Google Compute Engine or Google Container Engine. 

Next: Three ways to create a micro-service on Google Cloud - Part 2: Compute Engine

Docker on Mac

Tried to install DockerToolbox-1.8.2 on my MacBook Pro (running OS X10.8) last night.

But the DockerToolbox installer kept hanging. Googled around a bit. Tried a number of suggestions from various forums. None worked. Finally I noticed that most of the other users having this problem were also using OS X 10.8.

So, on a whim, I decided to update my Mac to Yosemite (OS X 10.10.5 to be specific). And that fixed the problem. I am now happilly running Docker on my MacBook Pro.

Over the next week or two, I'm sure to discover some apps that no longer work, probably while in the middle of a presentation. But at least docker works!

Best Source for Best Practices

In the enterprise software space, we have historically taken our cues from Enterprise Software Vendors. Vendors such as Microsoft, Sun, Oracle, IBM and BEA have hugely shaped the way we work.

This gave us things like:

• SOAP
• The J2EE App Server
• ASP.NET
• JNDI
• Enterprise Service Buss
• The word SOA Governance
• EJB
• JavaServer Faces

When I was new in the industry my techno-skepticism was not so well honed. I was guilty of spreading and promoting some bad ideas. I accepted everything the Enterprise Software Vendors said without question.

But about 10 years ago I started paying attention to what Google, Amazon, Facebook, Twitter and Netflix were doing. Turns out, many of them use Java. But usually in a different way. Often a simpler way.

If you want to learn enterprise best practices, I suggest you head on over to Google or Amazon and check out their cloud offerings. Study their APIs. Look at their HTTP services. Compare how they are doing things to the above list of Enterprise Mumbo Jumbo.

For example, look at what Java EE considers an appropriate stack for server-side apps. And compare that with Google's offering (Google App Engine Services). You will be surprised.

I saw a John Oliver special the other night that pointed out how much drug companies spend selling to doctors ($28 billion per year).

I suspect there has been something similar going on in enterprise software. If you go up the hill from where I live, to the really expensive houses, you will find a lot of folks with the title: Enterprise Software Sales. It seems to me that the corporate culture at these firms is to pay the Enterprise Software Salesperson the highest salary. The engineers live in the condos at the bottom of the hill, next to the garbage dump.

Dave Recommendation: we should start taking our cues (and best practices) from the great web companies instead of commercial enterprise software vendors.