The Idea

The entire world is fascinated by Chat GPT, and of course we at PROTOS have discussed it extensively as well. The more we talked about it, the more curious we became and it became clear that we wanted to find out how we could integrate Chat GPT into our daily workflow. Is Chat GPT a true cloud expert?
At our last team event, we immediately got to work. The idea behind it? We take a real customer request and give Chat GPT the tasks we normally work through after receiving an initial customer request.

We asked ourselves the following question:

To what extent can we use AI to solve a real customer project in AWS?

The Project:

Note: The charts and examples in this arcticle are in German

We received a fairly vague customer request via email, in which the customer requested several AWS services. Specifically, the customer requires a virtual machine on AWS with certain hardware specifications and a MariaDB database. The email did not include a description of the existing infrastructure and left us in the dark about how critical the new AWS services will be in terms of availability, security, etc.

Workflow:

How well in accordance with AWS best practices is the AI’s response to the underlying customer request:

• Prompt ChatGPT to provide a response to the customer’s query.

• The AI provided a relatively meaningful response in the form of an email that can easily be used for further communication with the customer. No assumptions were made about any existing infrastructure. The AI simply resolved the request and provided a specification for an EC2 instance and an RDS with MariaDB.
• The AI even provided the steps for an implementation of the given infrastructure, such as design, configuration, migration, and testing.
• After testing, the AI is able to add more services: We requested an automatic failover for the EC2 instance, and it suggested using an ELB along with Route53.
• The AI is good at responding to an initial customer request if it is not explicitly specific or relates to technologies that are unknown to the AI.

How well can the AI translate its initial response into code?

This is where it can get tricky: ChatGPT has no data after 2021. Therefore, some syntax it has provided for Terraform was no longer valid. However, it was smart enough to provide a valid VPC configuration with public and private subnets as well as the appropriate security groups. Interestingly, it has assumed that the EC2 instance will serve as a webserver since the requirement was to have it run CentOS.

Furthermore, ChatGPT seems to have forgotten about the RDS. Consequently, a security audit is critical when using ChatGPT to create an AWS infrastructure.

With the appropriate prompts (“add rds”, “use region eu-central”, etc.) it was able to create an infrastructure that covered the original customer requests. However, the response had syntactical errors as Terraform has made several updates since 2021.

There was even a cost estimate, which could serve as a very arbitrary value (deviation of 40-50%).

Conclusion

How far can we leverage AI to solve a real-world customer project in AWS?

Despite some vulnerabilities of Chat GPT, we were able to build a solid foundation for answering the customer’s query in less than an hour of work. The biggest issues were debugging the code generated by the AI and developing an understanding of when the AI cannot debug itself and a human developer should rely more on conventional debugging/research techniques.

For developers, this means that Chat GPT can be a useful tool to automate certain tasks related to customer requests in the cloud. It can help quickly deploy basic infrastructure and generate recommendations. However, developers need to be aware that the code generated by the AI may contain errors and may not always comply with current best practices. Therefore, it is important to carefully review, adjust, and debug the generated code. Human developers remain essential to validate the AI’s work, handle complex scenarios, and ensure a high-quality solution for customer projects in AWS.