Skip to Content
TensorPass
GuidesAI SDKs

LlamaIndex

Keep your LlamaIndex OpenAI LLM and its tool calls, and let it answer on the visitor's own AI account by passing the AiConnect client's options to the OpenAI SDK client it builds.

Your LlamaIndex code keeps the OpenAI LLM of @llamaindex/openai and its chat method with tools. The LLM takes the options of your AiConnect client, and every call then answers through the visitor's wallet, on the account the visitor picked for your site. That account can be a ChatGPT plan, a Claude plan, a DeepSeek key or any other account the wallet holds. The AI SDK guide explains how this works for every SDK.

The LlamaIndex packages for TypeScript are marked deprecated on npm since June 2026. The demo runs on their last releases.

Install

npm install @llamaindex/core @llamaindex/openai @tensorpass/aiconnect

Until @tensorpass/aiconnect is published on npm, the JavaScript quickstart shows how to point your project at a checkout of the repository.

The change

// Before
const llm = new OpenAI({ model: 'gpt-6-luna', apiKey: process.env.OPENAI_API_KEY });

// After
const llm = new OpenAI({
  model: 'gpt-6-luna',
  apiKey: aiconnect.sdkOptions.apiKey,
  additionalSessionOptions: aiconnect.sdkOptions,
});

aiconnect is your AiConnect client. additionalSessionOptions goes to the OpenAI SDK client the LLM builds, which then sends its requests through aiconnect.fetch. The LLM requires a key, so the placeholder aiwallet fills it, and the wallet never reads it.

The call runs in the page, where the wallet answers. Show your connect entry point first, as the React quickstart does: a request from a site the visitor has not connected fails with status 401.

Stopping an answer

LlamaIndex gives the OpenAI SDK client no abort signal. To let the visitor stop an answer, pass a fetch bound to your own signal:

const fetch: typeof aiconnect.fetch = (input, init) => (
  aiconnect.fetch(input, { ...init, signal: AbortSignal.any(init?.signal ? [signal, init.signal] : [signal]) })
);
const llm = new OpenAI({
  model: 'gpt-6-luna', apiKey: sdkOptions.apiKey, additionalSessionOptions: { ...sdkOptions, fetch },
});

Aborting signal then stops the answer in the wallet.

Chat with tools

The bookshop demo in apps/demo of the AI Connect repository runs this loop in its SDK mode. search_books stands for your own tools, runTool runs one and returns its result as text, and showText writes the answer as it streams.

import type {
  BaseTool, ChatMessage, PartialToolCall, ToolCall, ToolCallLLMMessageOptions,
} from '@llamaindex/core/llms';
import { OpenAI } from '@llamaindex/openai';

const { sdkOptions } = aiconnect;
const fetch: typeof aiconnect.fetch = (input, init) => (
  aiconnect.fetch(input, { ...init, signal: AbortSignal.any(init?.signal ? [signal, init.signal] : [signal]) })
);
const llm = new OpenAI({
  model: 'gpt-6-luna', apiKey: sdkOptions.apiKey, additionalSessionOptions: { ...sdkOptions, fetch },
});
const tools: BaseTool[] = [{ metadata: { name: 'search_books', description, parameters: searchBooksSchema } }];
// A call's input streams as text, then comes once more parsed when the call is complete.
const isComplete = (call: ToolCall | PartialToolCall): call is ToolCall => typeof call.input !== 'string';
const messages: ChatMessage<ToolCallLLMMessageOptions>[] = [
  { role: 'system', content: system },
  { role: 'user', content: question },
];
for (let step = 0; step < 4; step += 1) {
  let content = '';
  const calls = new Map<string, ToolCall>();
  for await (const chunk of await llm.chat({ messages, tools, stream: true })) {
    showText(chunk.delta);
    content += chunk.delta;
    const options = chunk.options ?? {};
    ('toolCall' in options ? options.toolCall : []).filter(isComplete).forEach((call) => calls.set(call.id, call));
  }
  if (!calls.size) {
    break;
  }
  messages.push({ role: 'assistant', content, options: { toolCall: [...calls.values()] } });
  for (const call of calls.values()) {
    const result = await runTool(call.name, call.input);
    messages.push({ role: 'user', content: result, options: { toolResult: { id: call.id, result, isError: false } } });
  }
}

The demo runs its own tool loop on llm.chat. The agent of @llamaindex/workflow runs on the same LLM, but a call that fails there is left unhandled and the agent's stream never ends. When your tools live on your MCP server, the MCP guide shows how to read them in the page.

The model that answers

The model you pass is ignored, since the visitor chooses the model in the wallet. The model names section tells which event carries the model that answers.

Errors and retries

A failure before any text throws the OpenAI SDK's error, with its status: 401 when the site is not connected, 403 when the visitor declined or left the wallet locked, 429 when the account's plan is spent. The errors section lists every status.

OpenAI from @llamaindex/openai retries a failed call ten times by default, so a spent plan keeps the visitor waiting. On Auto the wallet already moves to the visitor's next account, so pass maxRetries: 0 to answer at once.

What a request can carry

Images as data: URLs reach every account. Files, audio and OpenAI's own server tools are refused with a 400. The AI SDK guide lists what the wallet passes on to each account.

On this page