Overwatch is a pipeline that executes to aggregate and normalize all of the logs from all the supported sources and make them easy to interrogate for insights. The steps to deploying Overwatch are pretty simple but there are some specific details that may pertain to your deployment. Below are the top level steps that this section will walk you through the process.
When wanting to monitor your Workspaces, you might want to:
1. Configure Cloud Infrastructure
Select your cloud to find the details for cloud configuration here. (AWS | Azure | GCP)
2. Build / Apply Security
Reference Security Considerations page to help you build and apply a security model commensurate with your needs.
3. Configure Overwatch inputs
Reference the configuration page to clarify the configuration details and help you get started.
4. Run and Schedule the Job[s]
Decide whether you’d like to execute Overwatch as a JAR or a NOTEBOOK and schedule a job to periodically execute the job. It’s recommended to run Overwatch as a JAR as it unifies the deployment and doesn’t depend on another asset (the notebook) to secure and ensure no one edits, moves, or deletes it.
5. Productionize Your Deployment
Now that you’re live it’s important to optimize everything and ensure the data is safe in case something unexpected happens – see Productionizing
6. Share/Utilize Overwatch
Now you’re ready to onboard consumers across your workspaces. Details about how to do that can be found in the Sharing Overwatch page.
When adding a new workspace, all you have to do is add a new row in the configuration table for Overwatch containing the new workspace’s information. Please go through the previous steps for cloud infrastructure and security considerations to keep in mind to ensure the driver workspace can monitor the new workspace
As of 0.7.1.0 Overwatch is able to monitor remote workspaces but having one and only one global deployment often doesn’t meet customer needs and there are some requirements for a deployment to monitor remote workspaces. More details are available on the Security Considerations and Validations pages. The most important things to consider are:
If some of your workspaces do not meet these requirements; that’s ok, a local deployment with a simpler config will need to be created for those outliers. It’s completely fine to have a primary deployment that manages 20 workspaces and a few workspaces that have to have their own Overwatch Job. The data from each deployment can be dropped into the same or multiple targets, this is all part of the configuration.
Assume you have 23 workspaces, 20 of which meet the criteria discussed above and 3 that need to be locally configured. All 23 workspaces output their data into the same database (or multiple) but this would require that Overwatch run on 4 workspaces, 1 to manage the 20 and 1 on each of the 3 with special requirements.
Production can hold the data for all 23 workspaces even if some of the workspaces are deemed “non-prod”. This is ideal to allow an analyst to identify efficiency gains and metrics globally from a single location.
A non-prod Overwatch deployment is also recommended so that when new versions come out and/or schema upgrades come out your team can review and test the upgrade and update any dependent dashboards before you upgrade all 23 workspaces.
A non-prod Overwatch deployment typically consists of a subset of the 23 workspaces (usually 3-5 workspaces). This deployment will always be upgraded and validated before the global production deployment. This pattern enables Change Management best practices. Non-Prod and Prod can even be deployed on the same workspace just with two different configurations such that the non-prod has its own database names and storage locations.