A while back, I was speaking with a senior software engineer at New Century about how our development operations can keep up with the breakneck pace of change in our industry. He reasoned that software, as it is in both our job description and company name, should be the primary focus of what we do here. He's right, but great software is enabled by functioning infrastructure. This necessitates a solution to our problem that, succinctly stated, reads "but, it works on my machine!" This brings us to Chef, a configuration management platform.
What is Chef?
Chef, an open source software under the Apache License 2.0, is a configuration management tool written in Ruby. Chef operates on a server/client model to apply configurations and policies, organized in Recipes and Cookbooks, to a given node. These recipes are the building blocks of Chef's capabilities and can be written to meet a specific business need or taken from the Chef Supermarket, a community repository of cookbooks.
Each cookbook contains recipes and attributes. As a recipe is defined, a set of default attributes is defined, as well. These recipes define a series of actions that are performed to bring the node into the desired state. Attributes are a very powerful concept because they can be overridden in a hierarchical manner. Recipe attributes are overridden by role attributes are overridden by environmental attributes and on and on, allowing you to define a base state and override as needed.
Chef defines the notion of 'roles' and 'environments' to simplify deployment of resources. We know that every deployment of ArcGIS Enterprise needs ArcGIS for Server, Portal for ArcGIS, and Web Adaptor. Chef allows us to simplify deployments by defining a role which contains those recipes. We also know that deployments for development and production may require different configurations. Chef accomplishes this using environments. This allows us to, for instance, enable different logging levels in development and production environments.
What can Chef do for me?
Chef can manage complex infrastructure deployments in a reproducible manner, simply stated. Cookbooks, recipes, roles, environments, etc. are all contained in ruby and .json files. This betrays the fact that all of these entities are held in source control! Scriptable deployments are reproducible deployments and should be shared across an organization. With ESRI's release of the ArcGIS Cookbooks, much of the scripting and configuration has been done for you. ESRI provides cookbooks for its flagship ArcMap and ArcGIS Enterprise software, but also ArcPro, Web Adaptor (Tomcat and IIS), and many extensions for ArcGIS Enterprise. These cookbooks will install, configure, and authorize ESRI software.
Up to this point, I've rambled on about code and that, rightfully so, might lead you to believe that this is an immensely complex system. On the charge of complexity, I would plead guilty if not for the Chef Management Console!
The management console allows you to monitor and modify your Chef-maintained infrastructure without diving knuckle first into the code. And this is a guiding principle of the move – increasing complexity is a fact of the industry, but we can intelligently silo that complexity and manage the challenges associated. This struggle between necessary complexity and desired simplicity is two-fold; Infrastructure is complex and a difficult task to do correctly, but achieving this complexity requires the guise of simplicity.
How is New Century using Chef?
Our first foray into Chef will be internal ArcGIS for Pipeline Referencing (APR) deployments to support development work. What advantage does Chef give us in this task? Our APR environments require ArcMap, ArcPro, ArcGIS Enterprise, the APR extension and it's Event Editor web application. Add in SQL Server Database Server and Management Studio with two preliminary data models and you have quite the machine to configure.
As a developer at New Century, this is an exciting technology. We will be able to support our developers with more precise infrastructure configurations in a fraction of the time. If we encounter a problem during development, we won't bother to fix it. Why would we? Take that machine offline, deprovision it's resources while we bring a new, pristine deployment online. This speeds development time and insures that our infrastructure is functioning correctly. To our customers, this means faster response times on support and bug tickets.
I think the senior software engineer was more right than he knew in the sense that properly configured infrastructure should be something that you don't even notice.