While building a decentralized Twitter in a previous post, I included some code that implemented JSON persistence. With the launch of Meta’s Llama 3 this month, I thought it’d be a good opportunity to explore how a new LLM can help with coding.
Meta.AI is the chat frontend for Llama 3, which we will use after developing our persistence solution. As I write this (late April), Meta.AI is not available globally, so you may need to use a VPN to get to it. You may need to use a Facebook login for some things, like image generation, but that doesn’t seem to be a necessity.
First, I checked that I was actually talking to Llama 3, although this was not totally conclusive:
When looking at a previous LLM (Llama 2), I asked it to do some long multiplication that it failed to manage. It did however “fail like a human.” Let us see how this model has progressed:
A quick, and much better-looking answer. Still wrong; the answer is 1,223,834,880. However, we aren’t here to worry about this known lacuna. Let’s return to the main topic: persistence with JSON.
Saving with JSON
JSON is a portable data format, and while less efficient than an arbitrary native binary format, it can at least be checked by humans and implemented anywhere.
Because JSON can handle basic value types and some collections, we can in general map our instance data to JSON format. This is known as marshalling and is a lot less of an issue today than it used to be. Whether you want to persist (i.e., save and load) user preferences, a turn in a game or just some plain old useful data, JSON has your back.
In this post I’ll use .NET and C#. While we can do just fine with the inbuilt System.Text.Json, the Newtonsoft package is a bit nicer, so I’ll call that.
First I’ll set up a console project that will use the new style top-level statements to cut out the need for main statements and other boilerplate. So in a terminal:
I started with a quick Car class, which I wrote directly in Program.cs. We will use this as our target object:
Car car = new Car("Saab", "V90", Car.CarColor.Red);
Console.WriteLine($"First car: {car}");
public class Car
{
public enum CarColor {Red, Blue, White};
public string Brand;
public string Model;
public short MileageAtLastService;
public Car(string brand, string model, CarColor color)
{
this.Brand = brand;
this.Model = model;
this.MileageAtLastService = 0;
this.Color = color;
}
public void SetMileage(short mileage) => this.MileageAtLastService = mileage;
public override string ToString() => $"{Color} {Brand} {Model}" ;
}
The console output after running this is:
First car: Red Saab V90
To use the Newtonsoft.JSON package from the command line, we can just run this in the terminal:
JsonMarshalling> dotnet add package Newtonsoft.Json
You could also use the palette within VS Code, though I find that a little unreliable sometimes. So now that we have pushed the code for the car object to its own file, Car.cs, and namespace, CarClass, let’s do a simple conversion in Program.cs:
using CarClass;
using Newtonsoft.Json;
const string OUTFILE = @".cars.json";
Car car = new Car("Saab", "V90", Car.CarColor.Red);
Console.WriteLine($"First car: {car}");
string output = JsonConvert.SerializeObject(car);
Console.WriteLine($"Serialised: {output}");
The output to this in the terminal is:
Serialised: {"Brand":"Saab","Model":"V90","MileageAtLastService":0,"Color":0}
Oh, it didn’t work? But this is fine. The enum used for CarColor needs to be handled separately as it isn’t a type recognized by JSON. We could write the conversion ourselves or just leave it to the existing converter by decorating the property in Cars.cs:
using Newtonsoft.Json
using Newtonsoft.Json.Converters;
...
[JsonConverter(typeof(StringEnumConverter))]
public CarColor Color {get; set;}
...
The output result is now correct:
Serialised: {"Brand":"Saab","Model":"V90","MileageAtLastService":0,"Color":"Red"}
What we want to work toward for a round trip is to persist a couple of cars down to a JSON file, restore, and then update one aspect (for instance, a garage service) and save back, finally checking the JSON on file. First, let’s check that we can save a list down to a file:
using CarClass;
using Newtonsoft.Json;
const string OUTFILE = @"./cars.json";
List<Car> cars = new List<Car>();
Car car = new Car("Saab", "V90", Car.CarColor.Red);
Console.WriteLine($"First car: {car}");
cars.Add(car);
car = new Car("Volkswagen", "Polo", Car.CarColor.White);
Console.WriteLine($"Uncle's car: {car}");
cars.Add(car);
string output = JsonConvert.SerializeObject(cars);
File.WriteAllText(OUTFILE, output);
The file produced is fine, containing two JSON objects:
Let’s generalize the persistence part and put it in its own project file:
using CarClass;
using Newtonsoft.Json;
namespace PersistenceServices
{
public static class Persistence
{
const string JSONFILE = @"./cars.json";
public static void WriteCarsToFile(List<Car> cars)
{
WriteToFile(cars);
}
public static List<Car> ReadCarsFromFile()
{
return ReadFromFile<Car>(JSONFILE);
}
private static void WriteToFile<T>(List<T> list)
{
string jsonString = JsonConvert.SerializeObject(list, Formatting.Indented);
File.WriteAllText(JSONFILE, jsonString);
}
private static List<T> ReadFromFile<T>(string filename)
{
List<T>? list = null;
if (File.Exists(filename))
{
string jsonString = File.ReadAllText(filename);
list = JsonConvert.DeserializeObject<List<T>>(jsonString);
}
else Console.WriteLine("No existing file found for " + filename);
return list ?? new List<T>();
}
}
}
Note that the two methods that actually call JsonConvert do not need to know anything about the type being converted; they work with any generic list of types.
Here’s the round trip, which amends the second car’s service mileage:
using CarClass;
using Newtonsoft.Json;
using PersistenceServices;
List<Car> cars = new List<Car>();
Car car = new Car("Saab", "V90", Car.CarColor.Red);
Console.WriteLine($"First car: {car}");
cars.Add(car);
car = new Car("Volkswagen", "Polo", Car.CarColor.White);
Console.WriteLine($"Uncle's car: {car}"); cars.Add(car);
Persistence.WriteCarsToFile(cars);
List<Car> cars2 = Persistence.ReadCarsFromFile();
cars2.Last().MileageAtLastService = 2500;
Persistence.WriteCarsToFile(cars2);
The resulting file is even properly formatted:.
OK, so we created some persistence to a simple file using JSON, using a specific package. We ended up with these files in our project workspace:
Now let’s try to see how far we can get with Llama 3. Let’s break down what we did.
- Created a simple class to represent an owned car.
- Turned it into json.
- Used the json format to save and load into a local file.
So first let’s ask for a car class. The query is “Create a C# class that represents an owned car with these attributes: brand, make, miles since last service and color.”
Great. It also gave examples of using the class and setting the properties. Let us see if we can persuade it to make the color property an enum:
It did! Even with my spelling mistake in the request!
Now let’s see if it can use JSON to convert. Let’s force it to use Newtonsoft.json too.
It thinks this will be the output:
Of course, we already know this won’t work as it is. We need to fix the enum. We could try to tell Llama this, but we only know it isn’t correct because we’ve been here before. However, let’s take the win.
The good news is that Llama 3 was quick, well formatted and good with basic requirements.
- It allowed us to work on the code in a continuous fashion over time in one thread.
- It understood how to use an arbitrary named package.
- It correctly created example instances of Car, as well as some nice car colors.
- The answers are nicely formatted.
Where it went wrong was a detail that I happened to have worked through in advance. This leaves us in roughly the same place we’ve been in all the time with LLM coding: It can quickly create a working template that we can tailor, but it will have inaccuracies that will require fixing. This means it is a useful bootstrapping tool for a working developer but not yet a sort of magic solution maker.
The post Coding Test for Llama 3: Implementing JSON Persistence appeared first on The New Stack.
We put Meta's new Llama 3 large language model through its paces as a coding tool by asking it to create a JSON persistence solution.