Intro to Cloud Automation

Author: Emma Dionne

Ever since my first day on the job, I knew that this internship would teach me many new things that I had never learned about from my coding classes. Starting with a simple coding task, I jumped straight into salt and learned how to test code, address github issues, and submit a pull request. With the help of my AVID mentor, I learned about the basics of salt and its applications. There were also so many different resources on the salt project website that taught me about coding conventions and salt’s expansive capabilities.

Apart from the online resources the salt community provides, I also had the opportunity to sit in on a Salt Open Hour session. This experience was eye-opening in how collaborative the open-source and salt community is. I watched passionate coders share their work and saw how everyone uplifted the presenter and answered each other’s questions as thoroughly as possible.

Furthermore, the process of uploading code to the online repository was similar in its strive to better every coder through an extensive peer reviewing process. This environment was different to what I am used to in school, as collaboration is typically discouraged for coding assignments and assessments. Overall, I appreciate how this internship and learning how to use salt has taught me the true collaborative nature of coding, and it has inspired me to continue working hard on my computer science journey.

Prometheus

My first big project for this internship was modifying the pre-existing saltext-prometheus extension to use the prometheus client library. Originally, the code used a dictionary to store key and value pairs, but it needed to be transitioned to the prometheus client library. The original dictionary was replaced with a CollectorRegistry. To decide which data structure would replace the key-value pairs, I experimented with many different types to see which output would match the dictionary output the best: Gauge, Enum, Histogram, Info, Sample. In the end, Gauge was the data type to use because the value assigned to a Gauge could be more easily accessible than the other options. More information about the different options for adding the prometheus_client library can be found here: https://github.com/prometheus/client_python#node-exporter-textfile-collector

For example, when Info was implemented, the output was not of the same format as a dictionary:

i = Info('i_salt_version', 'Version of installed Salt package', registry = registry)
i.info({"val":"hi"})

 HELP i_salt_version_info Version of installed Salt package
# TYPE i_salt_version_info gauge
i_salt_version_info{val="hi"} 1.0

Similarly, the Enum syntax also was not a suitable option:

e = Enum('salt_version', 'Version of installed Salt package', states = ['highstate', 'version', 'two', 'three', 'four', 'five'], registry=registry)
e.state('four')
e.state('three')
e.state('five')
e.state('three')

# HELP salt_version Version of installed Salt package
# TYPE salt_version gauge
salt_version{salt_version="highstate"} 0.0
salt_version{salt_version="version"} 0.0
salt_version{salt_version="two"} 0.0
salt_version{salt_version="three"} 1.0
salt_version{salt_version="four"} 0.0
salt_version{salt_version="five"} 0.0

Even though the Gauge was the closest data type to a dictionary key-value pair, the Gauge’s value only took a float. If the inputted number was an integer or other numerical type, the Gauge would automatically typecast that value to a float. Furthermore, if the value of the Gauge is not of a numerical type, it throws an error. To combat this difference between the value type of a dictionary value vs. Gauge, some tests were modified to account for this difference.

"# HELP salt_failed Information regarding state with failure condition",
"# TYPE salt_failed gauge",
'salt_failed{state_comment="Command echo includeme run",state_id="echo includeme"} 1.0',
'salt_failed{state_comment="Command echo applyme run",state_id="echo applyme"} 1.0',

Azurerm

After the saltext-prometheus extension project wrapped up, I started creating the saltext-azurerm extension based off of the pre-existing saltext-azurearm extension. Since the libraries for saltext-azurearm were outdated, the code had to be updated to accommodate the newer dependency versions by using Azure SDK and PyPI to research about the new dependencies. One of the compatibility issues that were fixed was replacing the util dunders. Another change that was made was implementing azure-identity to replace azure-common clients. For example, ServicePrincipalCredentials was replaced with azure.identity.ClientSecretCredential and MSIAuthentication with azure.identity.DefaultAzureCredential. The azure-common credential options were too outdated and didn’t provide access to tokens anymore.

To test this new extension, every function of all resource types needed to be tested. Upon creating the different resources and running all the methods, smaller errors were encountered and fixed to accommodate.

/srv/salt/rg.sls

{%- set profile = {"subscription_id" : "5edf9949-aed3-4b3b-b450-14382a3d62aa"} %}


Ensure resource group exists:
    azurerm_resource.resource_group_present:
        - name: group1
        - location: eastus
        - tags:
            contact_name: Elmer Fudd Gantry
        - connection_auth: 
root@vm-emma:/srv/salt# salt-call azurerm_resource.resource_group_get group1 subscription_id=3287abc8-f98a-c678-3bde-326766fd3617

local:
    ----------
    id:
        /subscriptions/5edf9949-aed3-4b3b-b450-14382a3d62aa/resourceGroups/group1
    location:
        eastus
    name:
        group1
    properties:
        ----------
        provisioning_state:
            Succeeded
    tags:
        ----------
        contact_name:
            Elmer Fudd Gantry
    type:
        Microsoft.Resources/resourceGroups

The method works when it produces the expected result. In this example, it returned information about resource group1, which was expected.

Conclusion

Overall, I really enjoyed seeing the various applications of salt and being exposed to open source software. I think that salt is a great tool to use, not only for the physical code, but also for the community around it.