LLM-Powered Automatic Test Generation

A powerful feature of itdoc that automatically generates tests by analyzing source code through GPT.

Feature Overview

Is your project overloaded with APIs, making it hard to manually write test specs for each one using Itdoc?

const app = require("./expressApp")
const { describeAPI, itDoc, HttpStatus, field, HttpMethod } = require("itdoc")

describeAPI(
    HttpMethod.POST,
    "/signup",
    {
        summary: "User Registration API",
        tag: "Auth",
        description: "Registers a new user with username and password.",
    },
    app,
    (apiDoc) => {
        // Multiple test cases
        // ....
        // ....
        // ....
     });

Then try itdoc's LLM feature! This feature can automatically generate itdoc test code like the one above.

Let's compare before and after using the LLM feature.

🚫 Before (Manual)

Express App Source Code

app.js

const express = require("express")
const authenticationRouter = require("./authentication")

const app = express()

app.use(express.json())
app.use("/auth", authenticationRouter);
module.exports = app

Manual Test Code Writing Required

// Developers need to analyze and write manually
// Time-consuming and error-prone
// ...

✅ After (LLM Auto-generation)

With Just One Command

itdoc generate --app ./app.js

Automatically Generated Test Code

Generated Test File

const targetApp = app

describeAPI(
    HttpMethod.POST,
    "/signup",
    {
        summary: "User Registration API",
        tag: "Auth",
        description: "Registers a new user.",
    },
    targetApp,
    (apiDoc) => {
        itDoc("Returns 400 response when username is missing", async () => {
            await apiDoc
                .test()
                .req()
                .body({
                    password: field("Password", "password123"),
                })
                .res()
                .status(HttpStatus.BAD_REQUEST)
                .body({
                    error: field("Error message", "username is required"),
                })
        })

        itDoc("Returns 400 response when password is missing", async () => {
            await apiDoc
                .test()
                .req()
                .body({
                    username: field("Username", "newuser"),
                })
                .res()
                .status(HttpStatus.BAD_REQUEST)
                .body({
                    error: field("Error message", "password is required"),
                })
        })

        itDoc("Returns 400 response when password is less than 8 characters", async () => {
            await apiDoc
                .test()
                .req()
                .body({
                    username: field("Username", "newuser"),
                    password: field("Password", "short"),
                })
                .res()
                .status(HttpStatus.BAD_REQUEST)
                .body({
                    error: field("Error message", "password must be at least 8 characters"),
                })
        })

        itDoc("Returns 201 response when all conditions are met", async () => {
            await apiDoc
                .test()
                .req()
                .body({
                    username: field("Username", "newuser"),
                    password: field("Password", "password123"),
                })
                .res()
                .status(HttpStatus.CREATED)
        })
    }
)

Quickly adopt Itdoc with LLM functionality!

Requirements

• itdoc must be installed to use the CLI commands.

• This feature uses the ChatGPT API. Set up your OpenAI API key as follows:

  Add your key to the .env file like this

  echo OPENAI_API_KEY={ENTER_YOUR_GPT_KEY} > .env

  Alternatively, you can specify the path to a .env file that contains OPENAI_API_KEY as shown below:

  itdoc --e {YOUR_ENV_FILE_PATH}
  itdoc --env {YOUR_ENV_FILE_PATH}

API Key Security Warning

Add the .env file to .gitignore to prevent it from being uploaded to your Git repository

Content to add to .gitignore

.env

tip

• The .env file is searched in the current working directory by default. If it's in a different location, use the -e option to specify it.
• You can get an OpenAI API key from your OpenAI account.
• API usage incurs costs, so check the OpenAI pricing.
  • Usually, a small-sized app costs a one dollar.

Usage

Generating Tests from Source Code

Check First!

Currently, source code-based automatic test generation is only supported for the Express framework. If you're using a different framework, use the API Spec Markdown-based test generation method.

itdoc generate -a {YOUR_APP_FILE_PATH}
itdoc generate --app {YOUR_APP_FILE_PATH}

Option	Description
--app (-a)	Root app source code file path

When you run this command, it analyzes the Express application defined in {YOUR_APP_FILE_PATH} and automatically generates tests for the API endpoints in that application.