How to Build Generative AI Search for Your Data

Previously published on Nuclia.com. Nuclia is now Progress Agentic RAG.

Generative search is a much more powerful experience than traditional search. It allows users to ask questions and get answers in natural language.

When users need an answer, they have been trained to use a search engine. They type a few keywords, get a list of links, click on the links and try to find the answer.

So far, search engines have been doing half of the job. They have been finding the links, but not the answers.

Generative search is the missing piece of this puzzle.

In this regard, generative search is no longer a “search” at all. It is more like an “ask and answer.”

This is a much more intuitive way to interact with data, and it can be a game-changer for many applications.

There are now numerous solutions for building generative search, but they typically require a complex setup. Nuclia Retrieval-Augmented Generation (RAG) is a new kind of solution—one that is much easier to use.

How It Works

Nuclia offers a RAG-as-a-Service solution to build a generative search for your data. It covers data extraction from files, indexing and search. Additionally, it is thoughtfully designed to be user-friendly.

Ingestion

You do not need to pre-process your data, define a schema or train a model. You just need to upload your files directly to Nuclia, and it will take care of the rest.

If you have few files, you can upload them manually from the dashboard. If you want to integrate Nuclia RAG into your business data flow, then you can use our Software Development Kits (SDKs) to hook it up to your data sources.

If you are using JavaScript (or TypeScript), a typical ingestion hook would look like this:

import { Nuclia } from 'nuclia';
import { firstValueFrom, switchMap } from 'rxjs';

const nuclia = new Nuclia({
  backend: 'https://nuclia.cloud/api',
  zone: 'europe-1',
  knowledgeBox:  'YOUR-KB-ID',
  apiKey:  '<YOUR-API-KEY>',
});

async function ingestFile(id, filePath) {
  const file = await fs.readFile(filePath);
  await firstValueFrom(nuclia.getKnowledgeBox().pipe(
      switchMap((kb) => kb.createResource({ slug: id })),
      switchMap((resource) => resource.upload(file)),
    )
  );
}

ingestFile('doc1', 'path/to/your/file.pdf');
ingestFile('doc2', 'path/to/your/meeting.mp4');

If you prefer to use Python, you can use the following code:

from nuclia import sdk

KNOWLEDGE_BOX = "https://europe-1.nuclia.cloud/api/v1/kb/<YOUR-KB-ID>"
API_KEY = "<YOUR-API-KEY"

sdk.NucliaAuth().kb(url=KNOWLEDGE_BOX, token=API_KEY)

def upload_file(id, content_path):
    with open(content_path, "rb") as source_file:
        rid = sdk.NucliaResource().create(
            slug=id,
        )
        sdk.NucliaUpload().file(rid=rid, path=content_path, field="file")

upload_file("doc1", "path/to/your/file.pdf")
upload_file("doc2", "path/to/your/meeting.mp4")

Generative Search

Once your data is ingested (it may take few minutes, depending on the size of your data), you can start using it right away. You can ask questions and get answers in natural language.

In JavaScript, you can use the following code:

import { Nuclia } from 'nuclia';

const nuclia = new Nuclia({
  backend: 'https://nuclia.cloud/api',
  zone: 'europe-1',
  knowledgeBox: '',
  apiKey: '',
});

nuclia.knowledgeBox
  .ask('Is it possible for an Italian company to hire a salesperson in New Zealand?')
  .pipe(
    filter((response) => {
      if (response.incomplete) {
        console.log('Generating answer...');
      }
      return !response.incomplete;
    }),
  )
  .subscribe((answer) => {
    console.log(`Answer: ${answer.text}`);
    console.log('Sources:', answer.sources);
  });

The ask method, which is retrieving the generative answer from the Nuclia API, is very flexible.

For example, you can change the language model used for the answer, rather than using the default one defined in your Nuclia Knowledge Box settings:

nuclia.knowledgeBox.ask(
  'Is it possible for an Italian company to hire a salesperson in New Zealand?',
  undefined,
  undefined,
  { generative_model: 'claude-3' },
);

You can also customize the prompt used to generate the answer:

nuclia.knowledgeBox.ask(
  'Is it possible for an Italian company to hire a salesperson in New Zealand?',
  undefined,
  undefined,
  {
    prompt:
      'Given this context: {context}. Answer this {question} using the provided context. Please, answer always in French',
  },
);

Or provide extra context to the generative model:

nuclia.knowledgeBox.ask('Is it possible for an Italian company to hire a salesperson in New Zealand?', [
  { author: 'USER', text: 'Italy belongs to the European Union, the EU laws are applicable in Italy.' },
]);

What if You Are Not a Developer?

So, you think Python is a snake and JavaScript is a book editor in Indonesia? No worries, we have a no-code solution for you.

The Nuclia dashboard allows you to upload your files and generate a search widget that you can embed in your website just by copying and pasting a few lines of code.

The search widget editor is extremely flexible and will let you customize the generative search experience to fit your needs.

Would you like to try yourself? Sign up here.