GenAI Coding Experiments

Here’s your chance to experiment and evaluate GenAI (Generative AI) code generation, using ChatGPT or Claude.

You, or your team, will build and test a couple of simple tools.

If you’re in the meeting

If you’re at the BCS, your team is at your table. Share one laptop, do the exercises together, and discuss the results.

If you joined on Zooom

If you’re joining us remotely, you can work solo. If there are breakout rooms available in Zoom, you can join one of those.

Which tool should you use?

You’ll need to use a GenAI coding tool, and I recommend either OpenAI’s ChatGPT or Anthropic’s Claude Sonnet. To do that, someone will need to have an account, and you can quickly set up a free account if nobody has one available.

If you want to use ChatGPT or Claude Sonnet, and your team doesn’t have access to an existing account, you can sign up for a free account.

ChatGPT

OpenAI Sign-Up Process

  1. Visit chat.openai.com in your web browser
  2. Click on the “Sign up” button on the homepage
  3. Choose a sign-up method:
    1. Enter your email address
    2. Continue with Google account
    3. Continue with Microsoft account
    4. Continue with Apple account
  4. If using email, create a password and continue. Verify your email by clicking the link sent to your inbox. 1.Complete your profile by providing your full name and date of birth
  5. Agree to OpenAI’s Terms and Privacy Policy

Once registered, you’ll immediately have access to the free version of ChatGPT with GPT-4o mini.

Claude Sonnet

Free users of Anthropic’s Claude AI have enough access to the platform. Here are the key features and limitations of the free tier:

Access and Usage

  1. Free users can access Claude via the web interface, iOS app, and Android app
  2. Daily usage is limited to approximately 20 searches or messages per day
  3. The daily limit resets at midnight Pacific Time

Claude Sign-up process

To sign up for Claude AI, you need to follow these steps:

  1. Visit the official website at claude.ai or anthropic.com
  2. Click on the “Sign Up” or “Create Account” button on the homepage
  3. Choose a sign-up method:
    • Continue with Google account (fastest option)
    • Continue with Microsoft account
    • Continue with Apple account
    • Enter your email address and create a password
  4. If using email, verify your account by clicking the link sent to your inbox
  5. Complete your profile by providing your full name
  6. Verify your age (confirm you’re over 18) and agree to the Terms and Privacy Policy
  7. Optionally, verify your phone number for added security

Once registered, you’ll have immediate access to Claude AI with the Sonnet model available for free users.

Testing the code

I’d normally create automated tests for generated code, but for these exercises I recommend manual testing.

That means you’ll want to run the code that you’ve generated.

Don’t feel you have to finish the exercises

Don’t worry if you don’t have time to finish the exercises. Make sure you leave enough time to carry out a mini-retrospective before we reconvene.

If you want to, you can complete or tweak experiments after the meeting.

When we reconvene, I’ll be asking you to identify surprises, positive things, negative things, problems, and any questions you may have as a result of the experiments.

Remember that you can complete or tweak the experiments once the meeting is over.

Two experiments

Here are the two experiments.

Ideally, you or your team should come up with the prompts that you use for the exercises.

But if you get stuck, at the bottom are some prompts which I used which produced reasonable results.

Experiment one

We’ll start by defining the context for experiment one.

Imagine that you’re looking for a Python package on GitHub. You’ll find a PDF which describes a technique for evaluating GitHub projects. The goal of experiment one is to get Genai to write code to implement part of that process.

It’s best to implement it in Python if you can, but any other language would do.

Once you’ve generated the code, you can run it. Here’s a suitable project.

Experiment two

In Experiment 2, I’d like you to imagine that you’ve recently seen some mind maps and thought that they looked useful.

You’ve discovered that your colleagues use some free software called Freeplane and you’ve decided that you want to try and use GenAI to help you to generate mind maps from documents.

The experiment should be carried out in three stages.

Stage 1

In the first stage, you’ll explain that you want to create a freeplane mind map from YAML text, and ask GenAI to create a schema for the YAML document. YAML is a form of json designed to be easily readable by humans.

Like all json, YAML can be constrained by a schema. You’ll ask GenAI to create that schema. Make sure you save the file as you will need it later.

Stage 2

In the second stage, you’ll ask GenAI to convert a document into the YAML format which it created earlier.

I suggest you use this Markdown document for your test. You can download it and then upload it to ChatGPT or Claude.

Stage 3

In the third stage, you’ll ask GenAI to write code that converts the YAML file to a mind map file suitable for import into freeplane.

Run the code to turn the YAML file from the previous stage into a mind map file, You can check it by eye.: does it look as if it reflects the structure of the document?

If you have time, install and use FreePlane to load the file and see if it’s satisfactory.

Sample prompts

Use these if you get stuck.

Exercise one

Write Python code to help me evaluate a GitHub project. I’d like to know

  • When was the last update?
  • How many issues are there?
    • How many unresolved issues are there?
    • How long have they been around?
    • Are pull requests dealt with quickly?
  • How popular is the project?
    • How many people are watching it?
    • Is it often starred?
    • Has it often been forked?

(You can use https://github.com/romilly/rpi-docker-tensorflow as a sample project.)

Exercise two

Part one

I want to create a freeplane mind map from YAML text. Can you create a suitable json schema for the YAML file?

Part two

Here’s a markdown document (github-checklist.md): can you create a YAML file for it using that schema?

Part 3

Can you create a python function that will read a YAML file in the formayt spefied by the schema and create a .mm file that freeplane can load?