JSON to TOON Converter

What is TOON?

TOON (Token-Oriented Object Notation) is a compact data format designed to reduce token usage in Large Language Models (LLMs) by 30-60% compared to JSON. TOON maintains the same information as JSON but uses a more efficient representation, making it ideal for AI prompts, API calls, and any scenario where token count matters.

Why TOON Matters

• Cost reduction: Fewer tokens = lower API costs
• Context efficiency: More data fits in limited context windows
• Faster processing: Less data to process
• Better for LLMs: Optimized format for AI model consumption
• Maintains readability: Still human-readable unlike binary formats

TOON vs JSON Comparison

Size Comparison

JSON (156 characters):

{
  "users": [
    {"id": 1, "name": "Alice", "role": "admin"},
    {"id": 2, "name": "Bob", "role": "user"}
  ]
}

TOON (67 characters):

users[2]{id,name,role}:
  1,Alice,admin
  2,Bob,user

Savings: ~57% reduction in size

Token Usage

For LLMs, token count directly impacts cost:

• JSON: ~40 tokens
• TOON: ~17 tokens
• Savings: ~57% fewer tokens

TOON Syntax

Basic Structure

TOON uses a simplified syntax:

Objects:

key: value
nested:
  key: value

Arrays:

items:
  value1
  value2
  value3

Tabular Arrays (most efficient):

arrayName[count]{key1,key2,key3}:
  value1,value2,value3
  value4,value5,value6

Data Types

Tabular Array Format

The most efficient TOON feature is the tabular array format for arrays of objects with the same keys:

JSON:

{
  "users": [
    {"id": 1, "name": "Alice", "email": "alice@example.com"},
    {"id": 2, "name": "Bob", "email": "bob@example.com"},
    {"id": 3, "name": "Charlie", "email": "charlie@example.com"}
  ]
}

TOON (tabular):

users[3]{id,name,email}:
  1,Alice,alice@example.com
  2,Bob,bob@example.com
  3,Charlie,charlie@example.com

Benefits:

• Keys defined once (not repeated per object)
• Values only (no key names in data rows)
• Compact representation
• Easy to read and parse

TOON Format Examples

Simple Object

JSON:

{
  "name": "John",
  "age": 30,
  "city": "New York"
}

TOON:

name: John
age: 30
city: New York

Nested Objects

JSON:

{
  "user": {
    "name": "John",
    "address": {
      "street": "123 Main St",
      "city": "New York"
    }
  }
}

TOON:

user:
  name: John
  address:
    street: 123 Main St
    city: New York

Simple Array

JSON:

{
  "tags": ["javascript", "json", "toon"]
}

TOON:

tags:
  javascript
  json
  toon

Array of Objects (Tabular)

JSON:

{
  "products": [
    {"id": 1, "name": "Laptop", "price": 999.99},
    {"id": 2, "name": "Mouse", "price": 29.99},
    {"id": 3, "name": "Keyboard", "price": 79.99}
  ]
}

TOON:

products[3]{id,name,price}:
  1,Laptop,999.99
  2,Mouse,29.99
  3,Keyboard,79.99

Mixed Data Types

JSON:

{
  "config": {
    "enabled": true,
    "timeout": 30,
    "hosts": ["host1", "host2"],
    "metadata": null
  }
}

TOON:

config:
  enabled: true
  timeout: 30
  hosts:
    host1
    host2
  metadata: null

When to Use TOON

Use TOON When:

1. LLM Prompts: Sending data to AI models (ChatGPT, Claude, etc.)
2. API Calls: When token count affects cost or limits
3. Large Datasets: Arrays of objects with same structure
4. Cost Optimization: Reducing API costs
5. Context Windows: Maximizing data in limited context

Don't Use TOON When:

1. Human Configuration: JSON is more familiar
2. Complex Nested Structures: JSON may be clearer
3. Standard APIs: Most APIs expect JSON
4. Browser Storage: JSON is native in JavaScript
5. Interoperability: JSON is universally supported

Common Use Cases

1. LLM Prompts

Sending data to ChatGPT/Claude:

JSON (expensive):

{
  "users": [
    {"id": 1, "name": "Alice", "role": "admin", "email": "alice@example.com"},
    {"id": 2, "name": "Bob", "role": "user", "email": "bob@example.com"}
  ]
}

TOON (cost-effective):

users[2]{id,name,role,email}:
  1,Alice,admin,alice@example.com
  2,Bob,user,bob@example.com

Result: ~60% fewer tokens, significant cost savings

2. API Response Optimization

Before (JSON):

{
  "status": "success",
  "data": {
    "items": [
      {"id": 1, "title": "Item 1", "price": 10.99},
      {"id": 2, "title": "Item 2", "price": 20.99}
    ]
  }
}

After (TOON):

status: success
data:
  items[2]{id,title,price}:
    1,Item 1,10.99
    2,Item 2,20.99

3. Configuration Files

Terraform-like config in TOON:

resources[3]{type,name,config}:
  aws_s3_bucket,my-bucket,region:us-east-1
  aws_ec2_instance,web-server,type:t3.medium
  aws_rds_instance,db,engine:postgres

4. Log Aggregation

Structured logs in TOON:

logs[3]{timestamp,level,message,service}:
  2024-01-01T12:00:00Z,info,Request received,api
  2024-01-01T12:00:01Z,error,Database timeout,api
  2024-01-01T12:00:02Z,info,Request completed,api

5. Data Export for AI

Exporting database results for AI analysis:

JSON:

{
  "sales": [
    {"date": "2024-01-01", "amount": 1000, "product": "Widget"},
    {"date": "2024-01-02", "amount": 1500, "product": "Gadget"}
  ]
}

TOON:

sales[2]{date,amount,product}:
  2024-01-01,1000,Widget
  2024-01-02,1500,Gadget

TOON Syntax Rules

1. Key-Value Pairs

Format: key: value

Rules:

• Keys are unquoted identifiers
• Values can be strings, numbers, booleans, null
• No quotes needed for simple strings
• Escape special characters if needed

2. Nested Objects

Format:

parent:
  child: value
  nested:
    key: value

Rules:

• Use 2-space indentation
• Child keys are indented relative to parent

3. Arrays

Simple arrays:

items:
  value1
  value2

Tabular arrays:

arrayName[count]{key1,key2}:
  value1,value2
  value3,value4

Rules:

• Tabular format requires same keys for all objects
• Count must match actual number of rows
• Values separated by commas
• Rows separated by newlines

4. Escaping

Special characters to escape:

• Comma (,): \,
• Colon (:): \:
• Newline (\n): \\n

Example:

message: Hello\, world
path: C\:\\Users\\Name
text: Line 1\\nLine 2

Conversion Examples

Example 1: User List

JSON:

{
  "users": [
    {"id": 1, "name": "Alice", "email": "alice@example.com", "active": true},
    {"id": 2, "name": "Bob", "email": "bob@example.com", "active": false}
  ]
}

TOON:

users[2]{id,name,email,active}:
  1,Alice,alice@example.com,true
  2,Bob,bob@example.com,false

Size reduction: 156 → 67 characters (57%)

Example 2: Complex Nested Structure

JSON:

{
  "company": {
    "name": "Acme Corp",
    "employees": [
      {"id": 1, "name": "John", "department": "Engineering"},
      {"id": 2, "name": "Jane", "department": "Marketing"}
    ],
    "settings": {
      "timezone": "UTC",
      "language": "en"
    }
  }
}

TOON:

company:
  name: Acme Corp
  employees[2]{id,name,department}:
    1,John,Engineering
    2,Jane,Marketing
  settings:
    timezone: UTC
    language: en

Example 3: Mixed Arrays

JSON:

{
  "data": {
    "numbers": [1, 2, 3],
    "users": [
      {"id": 1, "name": "Alice"},
      {"id": 2, "name": "Bob"}
    ],
    "tags": ["tag1", "tag2"]
  }
}

TOON:

data:
  numbers:
    1
    2
    3
  users[2]{id,name}:
    1,Alice
    2,Bob
  tags:
    tag1
    tag2

Best Practices

1. Use Tabular Format for Arrays

When you have arrays of objects with the same keys, always use tabular format:

Good:

users[3]{id,name,email}:
  1,Alice,alice@example.com
  2,Bob,bob@example.com
  3,Charlie,charlie@example.com

Less efficient:

users:
  id: 1
  name: Alice
  email: alice@example.com
  id: 2
  name: Bob
  email: bob@example.com

2. Keep Keys Short

Shorter keys = fewer tokens:

Good:

u[3]{id,nm,em}:
  1,Alice,alice@example.com

Acceptable (more readable):

users[3]{id,name,email}:
  1,Alice,alice@example.com

3. Consistent Formatting

Use consistent indentation (2 spaces):

parent:
  child: value
  nested:
    key: value

4. Validate Count

Ensure array count matches actual rows:

items[3]{id,name}:  // Must have exactly 3 rows
  1,Item 1
  2,Item 2
  3,Item 3

5. Escape When Needed

Escape special characters in values:

message: Hello\, world
path: C\:\\Users

Token Savings Calculation

How to Calculate Savings

Formula:

Savings % = (1 - (TOON tokens / JSON tokens)) × 100

Example:

• JSON: 100 tokens
• TOON: 40 tokens
• Savings: (1 - 40/100) × 100 = 60%

Factors Affecting Savings

1. Array size: Larger arrays = more savings
2. Key length: Shorter keys = more savings
3. Structure: Tabular arrays = maximum savings
4. Nesting: Deeper nesting = less savings

Typical Savings

Common Mistakes to Avoid

1. Incorrect Count

Wrong:

users[3]{id,name}:  // Says 3 but only 2 rows
  1,Alice
  2,Bob

Correct:

users[2]{id,name}:
  1,Alice
  2,Bob

2. Missing Escaping

Wrong:

message: Hello, world  // Comma breaks parsing

Correct:

message: Hello\, world

3. Inconsistent Keys

Wrong (can't use tabular format):

users[2]{id,name}:
  1,Alice,admin  // Has 3 values but only 2 keys

Correct:

users[2]{id,name,role}:
  1,Alice,admin

4. Wrong Indentation

Wrong:

parent:
child: value  // Missing indentation

Correct:

parent:
  child: value

5. Not Using Tabular Format

Inefficient:

users:
  id: 1
  name: Alice
  id: 2
  name: Bob

Efficient:

users[2]{id,name}:
  1,Alice
  2,Bob

TOON vs Other Formats

TOON vs JSON

TOON vs CSV

TOON vs YAML

Frequently Asked Questions

What is the main advantage of TOON?

Token efficiency: TOON reduces token usage by 30-60% compared to JSON, making it ideal for LLM prompts and API calls where token count directly impacts cost.

Can TOON represent all JSON structures?

Mostly yes, but some complex nested structures may be less efficient. Tabular arrays (arrays of objects with same keys) see the biggest benefits.

Is TOON a standard format?

No, TOON is a custom format designed specifically for token efficiency. JSON remains the standard for most use cases.

When should I use TOON instead of JSON?

Use TOON when:

• Sending data to LLMs (ChatGPT, Claude, etc.)
• Token count affects cost or limits
• You have arrays of objects with the same structure
• You need to maximize data in limited context windows

How do I parse TOON?

TOON can be parsed and converted back to JSON. The format is designed to be:

• Human-readable
• Easy to parse
• Reversible (TOON ↔ JSON)

Does TOON support comments?

No, TOON does not support comments. If you need comments, use JSON or YAML.

Can I use TOON in production APIs?

Generally no, unless your API specifically supports TOON. Most APIs expect JSON. TOON is best for:

• LLM prompts
• Internal data processing
• Cost-optimized data transfer

How much can I save with TOON?

Typical savings:

• Tabular arrays: 50-70% token reduction
• Simple objects: 20-40% token reduction
• Mixed structures: 30-60% token reduction

Actual savings depend on your data structure.

Additional Resources

• JSON Specification (RFC 8259)
• OpenAI Token Usage Guide
• Anthropic Claude Token Counting
• Token Optimization Strategies