Comments on: How to create a private ChatGPT that interacts with your local documents

By: Tristan

Tristan — Wed, 14 Aug 2024 15:17:51 +0000

This is a RAG model of LLM a very simplified version of an LLM. Essentially a very enhanced search engine with natural language input and output. Useful but not really AI.

By: Michal

Michal — Wed, 11 Oct 2023 14:43:51 +0000

Hey,
do any of those models handle inputs in various languages (other then English)? I wonder if the model will be able to understand the knowledge documents e.g. in Polish, Spanish and English and compile them into an answer in English.

By: Ben Dickson

Ben Dickson — Fri, 04 Aug 2023 05:50:42 +0000

In reply to Vitaly Karasik.

Hi Vitaly,
Good questions!
1) The embedding model used in this project is the SentenceTransformer from Hugging Face. It is independent of the LLM.
2) If you replace the LLM, you do not need to ingest the documents again. But if you change your embedding model, you have to do so. The key is to use the same model to 1) embed the documents and store them in the vector DB and 2) embed user prompts to retrieve documents from the vector DB.
Hope this helps!

By: Vitaly Karasik

Vitaly Karasik — Thu, 03 Aug 2023 03:24:48 +0000

Many thanks!
Two questions:
1) Regarding “2. Embedding model’ – I understand that Open-source LLM is pluggable. What about Embedding model? Where is coming from? Is it different between GPT4all and PrivateGPT for example?
2) If I replace LLM model, should I ingest my documents again?

By: Ian Skidmore

Ian Skidmore — Fri, 07 Jul 2023 16:22:24 +0000

I followed the instructions in the project README and ingested the example ‘state of the union’ text file, but when I tried to run the privateGPT script, I got the following error:
Found model file at models/ggml-gpt4all-j-v1.3-groovy.bin
Invalid model file
Traceback (most recent call last):
File “C:\Users\Ian\python\privateGPT\privateGPT.py”, line 83, in
main()
File “C:\Users\Ian\python\privateGPT\privateGPT.py”, line 38, in main
llm = GPT4All(model=model_path, n_ctx=model_n_ctx, backend=’gptj’, n_batch=model_n_batch, callbacks=callbacks, verbose=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “pydantic\main.py”, line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for GPT4All
__root__
Unable to instantiate model (type=value_error)

I’m running on a Windows machine – I presume that the .bin file format is correct? Or is there a specific binary version for Windows that I need to access?

By: Ben Dickson

Ben Dickson — Wed, 05 Jul 2023 19:36:50 +0000

In reply to David. Yes. But you have to update their record in the vector DB every time the file's content is modified.

By: David

David — Mon, 03 Jul 2023 14:46:41 +0000

Can pages that are constantly being edited be stored as a source in the database?

By: Bobf

Bobf — Sat, 01 Jul 2023 21:08:43 +0000

In reply to Sam P. When you clone the PrivateGPT repo, it includes a file, example.env. You need to copy that file to .env in that same folder. This will set the required environment variables for ingest.py

By: Arthur

Arthur — Sat, 17 Jun 2023 12:43:03 +0000

In reply to Andrew. Yeah, in Fact, Google announced that you would be able to query anything stored within one’s google drive. I expect it will be much more seamless, albeit, your documents will all be avail to Google and your number of queries may be limited each day or every couple of hours. Microsoft was also doing somehting like this, but I didn’t hear the explicit details. I know it is to be available for companies to better look up policy and procedure etc. But I can’t recall if they were wanting do Natural Language Queries (nlq) for the average consumer.

By: josh kincannon

josh kincannon — Wed, 14 Jun 2023 02:48:10 +0000

I am trying to use this to review medical records and have the program answer questions about those records for civil litigation, i’m a noob. I got it running with the groovy bin model and also vicuna 13b-q5_1. I have a powerful pc, 13th gen intel I9, 96gb ram. I have put my records, about 125 pages, into source folder, in pdf. I ask it the patient name, and I time it’s response. I’ve used both models, and tried putting the data in different format. it almost got the answer correct the first time, every time since then I don’t get the correct response, it hiccups and loops, however when it is citing to the page where it got the info it list the correct response in that data, but it did not properly extract and give it as an answer. Same with height and weight. I believe this is a problem with the underlying model, and that if I used a clinical language model like gatortron it might work better, but I can’t get gatortron to work. Should the system as I’ve described it with the models I have installed work better at extracting the proper answers to my simple questions about data contained in a pdf?

By: Sam P

Sam P — Mon, 05 Jun 2023 18:17:32 +0000

It seems like an environment variable called “persist_directory” needs to be set. I tried setting it to db and it helped me get through to the next error message(!)

By: Ben Dickson

Ben Dickson — Mon, 05 Jun 2023 07:59:13 +0000

In reply to Vermandel Michiel. This is interesting. Can you share a bit more? Which model are you using? Also, how many documents do you have and how long are they on general?

By: Vermandel Michiel

Vermandel Michiel — Mon, 05 Jun 2023 06:47:31 +0000

I followed your instructions, and everything is working… sort of… PrivateGPT seems to retrieve the document(s) that fit the query the best and then just returns that document, instead of interpreting it and using it to build an appropriate answer.

By: Fred

Fred — Sun, 04 Jun 2023 20:47:20 +0000

You can also simply install FractalGPT and use YOUR private Azure OpenAI subscription.

By: Ben Dickson

Ben Dickson — Sun, 04 Jun 2023 05:49:16 +0000

In reply to Raheel Ali Khan. No. You just need to update your vector database.

By: Andrew Andreyev

Andrew Andreyev — Sat, 03 Jun 2023 21:58:58 +0000

Great summary of a complex workflow. Appreciated!

By: Raheel Ali Khan

Raheel Ali Khan — Sat, 03 Jun 2023 21:33:12 +0000

Do we need to re-train the model if more documents get added in the repository after initial model? I have a growing repository of all types of documents such as; Google docs, sheets and slides etc. Will this private GPT work for all those docs too?

By: Ben Dickson

Ben Dickson — Sat, 03 Jun 2023 19:26:30 +0000

In reply to nw. You're probably having problem with the Python version and libraries. I suggest your install Anaconda, which already has the right Python version and libraries. And then try to install PrivateGPT in Anaconda. It should be smoother.

By: nw

nw — Sat, 03 Jun 2023 18:43:05 +0000

It is great to see this content however for us lesser tech monkeys I am getting errors when I copy and paste your lines in Python. Can you troubleshoot this? I am a complete noob to this but what you mention, running it on my own machine, teaching it myself and expanding capabilities is pure gold.

By: Andrew

Andrew — Sat, 03 Jun 2023 17:01:08 +0000

This makes a great deal of sense and seems practical. I’m still hesitant to pull the trigger, mainly because I’m pretty confident Microsoft and other bigger tech companies will be able to do this in a much more seamless manner in the very near future.

By: Richard Ametih

Richard Ametih — Sat, 03 Jun 2023 06:23:49 +0000

One of the most useful articles on how chat GPT works. I’m a urologist taking courses full stack software developer. Looking for ways harness the power of A.I for simplified patients doctor interaction

By: Oktrik

Oktrik — Fri, 02 Jun 2023 16:59:00 +0000

In reply to XAVIER DIOKNO. It's already been answered here "They have also been designed to run on computers with consumer-grade hardware"

By: XAVIER DIOKNO

XAVIER DIOKNO — Thu, 01 Jun 2023 15:03:11 +0000

Thank you for providing this helpful content. I did have a question. If you are hosting your own LLM, won’t that require a substantial amount of computing resources to support the application? I.e., the CPU needed to handle multiple user chats?