I’ve had pretty good luck when having it write Python automation scripts around AWS using Boto3.
If it’s a newer API that ChatGPT isn’t trained on, I would either tell it where to find the newest documentation for the API on the web or paste the documentation in.
It usually worked pretty well.
If the author of the library wrote good documentation and sample code, you wouldn’t need StackOverflow hypothetically if ChatGPT was trained on it
Apple is training its own autocomplete for Swift on its documentation and its own sample code.
We don't have to guess. Just look at languages which have been around for a while, achieved some baseline level of popularity to have a decent amount of public code available, like Elixir.
I haven't found an LLM that could reliably produce syntactically correct code, much less logically correct code.
Since LLMs have been a thing, I’ve been heavily involved in the AWS ecosystem and automation.
ChatGPT is well trained on the AWS SDK for various languages. I can usually ask it to do something that would be around up to a 100-200 line Python script and it gets it correct. Especially once it got web search capabilities, I could tell it to “verify Boto3 (the AWS SDK for Python) functions on the web”.
I’ve also used it to convert some of my handwritten AWs SDK based scripts between languages depending on the preferences of the client - C#, Python, JavaScript and Java.
It also does pretty well at converting CloudFormation to idiomatic CDK and Terraform.
I was going into one project to teach the client how to create deployment pipelines for Java based apps to Lambda, EC2 and ECS (AWS’s Docker orchestration service).
I didn’t want to use their code for the proof of concept/MVP. But I did want to deploy a sample Java API. I hadn’t touch Java in over 20 years. I was a C#/Python/Node/(barely) Go developer.
I used ChatGPT to create a sample CRUD API in Java that connected to a database. It worked perfectly. I also asked about proper directory structure.
It didn’t work perfectly with helping me build the Docker container. But it did help.
On another note: it’s not too much of leap to see how Visual Studio or ReSharper could integrate an LLM better and with static language, guarantee that the
code is at least syntactically correct and the functions that are call exist in the standard library or the solution.
They can already do quick, real time warnings and errors if your code won’t compile as you type.
