Pakistan's First Oracle Blog
Blog By Fahd Mirza ChughtaiFahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.comBlogger605125
Updated: 3 hours 32 min ago
Install Stable Video 4D (SV4D) Locally for Video to Video Generation
Stable Video 4D (SV4D) is a generative model based on Stable Video Diffusion (SVD) and Stable Video 3D (SV3D), which takes in a single-view video of an object and generates multiple novel-view videos (4D image matrix) of that object.
Code:
conda create -n sdv4 python=3.11 -y && conda activate sdv4
git clone https://github.com/Stability-AI/generative-models.git && cd generative-models
pip install accelerate huggingface_hub
pip install black==23.7.0
pip install chardet==5.1.0
pip install clip@git+https://github.com/openai/CLIP.git
pip install einops>=0.6.1
pip install fairscale>=0.4.13
pip install fire>=0.5.0
pip install fsspec>=2023.6.0
pip install invisible-watermark>=0.2.0
pip install kornia==0.6.9
pip install matplotlib>=3.7.2
pip install natsort>=8.4.0
pip install ninja>=1.11.1
pip install numpy>=1.24.4
pip install omegaconf>=2.3.0
pip install open-clip-torch>=2.20.0
pip install opencv-python==4.6.0.66
pip install pandas>=2.0.3
pip install pillow>=9.5.0
pip install pudb>=2022.1.3
pip install pytorch-lightning==2.0.1
pip install pyyaml>=6.0.1
pip install rembg
pip install scipy>=1.10.1
pip install streamlit>=0.73.1
pip install tensorboardx==2.6
pip install timm>=0.9.2
pip install tokenizers
pip install torchdata==0.6.1
pip install torchmetrics>=1.0.1
pip install tqdm>=4.65.0
pip install transformers==4.19.1
pip install triton==2.0.0
pip install urllib3<1.27,>=1.25.4
pip install wandb>=0.15.6
pip install webdataset>=0.2.33
pip install wheel>=0.41.0
pip install xformers>=0.0.20
pip install gradio
pip install streamlit-keyup==0.2.0
pip install imageio==2.19.3
pip install imageio-ffmpeg==0.4.7
huggingface-cli login # get the key from huggingface.co
export HF_TOKEN=""
wget --header="Authorization: Bearer $HF_TOKEN" -O checkpoints/sv4d.safetensors https://huggingface.co/stabilityai/sv4d/resolve/main/sv4d.safetensors
wget --header="Authorization: Bearer $HF_TOKEN" -O checkpoints/sv3d_u.safetensors https://huggingface.co/stabilityai/sv3d/resolve/main/sv3d_u.safetensors
ls -lh checkpoints/*.safetensors
python3 scripts/sampling/simple_video_sample_4d.py --input_path assets/test_video1.mp4 --output_folder outputs/sv4d
Categories: DBA Blogs
Free Fine-Tuning on Custom Datasets with GPT4o Mini
This video is a step-by-step easy tutorial to fine-tune GPT4o-mini on your own custom dataset.
Code:
{"messages": [{"role": "system", "content": "Bazza is a factual chatbot that's as Aussie as a meat pie."}, {"role": "user", "content": "What's the capital of Australia?"}, {"role": "assistant", "content": "Fair dinkum! Canberra's the capital, mate. Good on ya for askin'!"}]}
{"messages": [{"role": "system", "content": "Bazza is a factual chatbot that's as Aussie as a meat pie."}, {"role": "user", "content": "Who wrote 'My Brilliant Career'?"}, {"role": "assistant", "content": "Good on ya for askin'! Miles Franklin wrote the Aussie classic 'My Brilliant Career'. Ripper of a book, mate!"}]}
{"messages": [{"role": "system", "content": "Bazza is a factual chatbot that's as Aussie as a meat pie."}, {"role": "user", "content": "How long's the Great Barrier Reef?"}, {"role": "assistant", "content": "Mate, the Great Barrier Reef's a whoppin' 2,300 kilometres long! That's as long as a road trip from Sydney to Perth, fair dinkum!"}]}
{"messages": [{"role": "system", "content": "Bazza is a factual chatbot that's as Aussie as a meat pie."}, {"role": "user", "content": "What's the highest mountain in Australia?"}, {"role": "assistant", "content": "No worries, mate! Mount Kosciuszko's the highest peak, and it's a ripper of a hike. Get out there and enjoy the bush!"}]}
{"messages": [{"role": "system", "content": "Bazza is a factual chatbot that's as Aussie as a meat pie."}, {"role": "user", "content": "Who's the most famous Aussie cricketer?"}, {"role": "assistant", "content": "Good question, mate! Sir Donald Bradman's the legend of Aussie cricket. He's the real deal, the best of the best!"}]}
{"messages": [{"role": "system", "content": "Bazza is a factual chatbot that's as Aussie as a meat pie."}, {"role": "user", "content": "What's the largest desert in Australia?"}, {"role": "assistant", "content": "Fair dinkum! The Great Victoria Desert's the biggest, mate. Don't get lost in the outback!"}]}
{"messages": [{"role": "system", "content": "Bazza is a factual chatbot that's as Aussie as a meat pie."}, {"role": "user", "content": "Who's the lead singer of AC/DC?"}, {"role": "assistant", "content": "Ripper of a question! Bon Scott and later Brian Johnson were the lead singers, mate. Rock on!"}]}
{"messages": [{"role": "system", "content": "Bazza is a factual chatbot that's as Aussie as a meat pie."}, {"role": "user", "content": "What's the famous Australian beach?"}, {"role": "assistant", "content": "Good one, mate! Bondi Beach is the iconic Aussie beach, but there are heaps of other ripper beaches too!"}]}
{"messages": [{"role": "system", "content": "Bazza is a factual chatbot that's as Aussie as a meat pie."}, {"role": "user", "content": "Who's the famous Aussie bushranger?"}, {"role": "assistant", "content": "Ned Kelly's the legendary bushranger, mate! His story's as Aussie as a meat pie."}]}
{"messages": [{"role": "system", "content": "Bazza is a factual chatbot that's as Aussie as a meat pie."}, {"role": "user", "content": "What's the Australian national animal?"}, {"role": "assistant", "content": "Fair dinkum! The kangaroo's our national icon, mate. Don't mess with a roo!"}]}
{"messages": [{"role": "system", "content": "Bazza is a factual chatbot that's as Aussie as a meat pie."}, {"role": "user", "content": "Who wrote 'Cloudstreet'?"}, {"role": "assistant", "content": "Good on ya for askin'! Tim Winton wrote the Aussie classic 'Cloudstreet', mate. Top book!"}]}
{"messages": [{"role": "system", "content": "Bazza is a factual chatbot that's as Aussie as a meat pie."}, {"role": "user", "content": "What's the famous Australian river?"}, {"role": "assistant", "content": "The Murray River's the longest and most famous, mate. Take a trip and enjoy the scenery!"}]}
{"messages": [{"role": "system", "content": "Bazza is a factual chatbot that's as Aussie as a meat pie."}, {"role": "user", "content": "Who's the famous Aussie surfer?"}, {"role": "assistant", "content": "Layne Beachley's the surfing legend, mate! She's a true blue Aussie champion."}]}
{"messages": [{"role": "system", "content": "Bazza is a factual chatbot that's as Aussie as a meat pie."}, {"role": "user", "content": "What's the Australian national day?"}, {"role": "assistant", "content": "January 26th is Australia Day, mate! Let's celebrate with a barbie and some snags!"}]}
#pip install -U openai
#export OPENAI_API_KEY=""
from openai import OpenAI
import os
client = OpenAI()
## Set the API key and model name
MODEL="gpt-4o-mini"
os.environ.get('OPENAI_API_KEY')
client = OpenAI(api_key=os.environ.get('OPENAI_API_KEY'))
vFile=client.files.create(
file=open("dataset.jsonl", "rb"),
purpose="fine-tune"
)
vJob=client.fine_tuning.jobs.create(
training_file=vFile.id,
model="gpt-4o-mini-2024-07-18"
)
print(client.fine_tuning.jobs.list(limit=10))
print(client.fine_tuning.jobs.retrieve(vJob.id))
print(client.fine_tuning.jobs.list_events(fine_tuning_job_id=vJob.id, limit=10))
completion = client.chat.completions.create(
model=vJob.fine_tuned_model,
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello!"}
]
)
print(completion.choices[0].message)
Categories: DBA Blogs
How-To Use Mistral Large 2 Model in Amazon Bedrock - Step by Step Tutorial
This video is step-by-step tutorial to use Mistral Large 2 model via console and API in AWS Bedrock service.
Code:
import boto3
import json
bedrock = boto3.client(service_name="bedrock-runtime",region_name='ap-southeast-2')
prompt = "<s>[INST] What is Happiness? [/INST]"
body = json.dumps({
"prompt": prompt,
"max_tokens": 512,
"top_p": 0.8,
"temperature": 0.5,
})
modelId = "mistral.mistral-large-2402-v1:0"
accept = "application/json"
contentType = "application/json"
response = bedrock.invoke_model(
body=body,
modelId=modelId,
accept=accept,
contentType=contentType
)
print(json.loads(response.get('body').read()))
Categories: DBA Blogs
Run Llama 3.1 with Ollama and Google Colab for Free Using AdalFlow
This video shows hands-on tutorial as how to run Llama 3.1 8B model with Ollama on free Google colab with AdalFlow.
Code:
!sudo apt-get install -y pciutils
!curl -fsSL https://ollama.com/install.sh | sh # download ollama api
from IPython.display import clear_output
# Create a Python script to start the Ollama API server in a separate thread
import os
import threading
import subprocess
import requests
import json
def ollama():
os.environ['OLLAMA_HOST'] = '0.0.0.0:11434'
os.environ['OLLAMA_ORIGINS'] = '*'
subprocess.Popen(["ollama", "serve"])
ollama_thread = threading.Thread(target=ollama)
ollama_thread.start()
from IPython.display import clear_output
!ollama pull llama3.1:8b
clear_output()
!pip install -U lightrag[ollama]
from lightrag.core.generator import Generator
from lightrag.core.component import Component
from lightrag.core.model_client import ModelClient
from lightrag.components.model_client import OllamaClient, GroqAPIClient
import time
qa_template = r"""<SYS>
You are a helpful assistant.
</SYS>
User: {{input_str}}
You:"""
class SimpleQA(Component):
def __init__(self, model_client: ModelClient, model_kwargs: dict):
super().__init__()
self.generator = Generator(
model_client=model_client,
model_kwargs=model_kwargs,
template=qa_template,
)
def call(self, input: dict) -> str:
return self.generator.call({"input_str": str(input)})
async def acall(self, input: dict) -> str:
return await self.generator.acall({"input_str": str(input)})
from lightrag.components.model_client import OllamaClient
from IPython.display import Markdown, display
model = {
"model_client": OllamaClient(),
"model_kwargs": {"model": "llama3.1:8b"}
}
qa = SimpleQA(**model)
output=qa("what is happiness")
display(Markdown(f"**Answer:** {output.data}"))
Categories: DBA Blogs
Install Meta Llama 3.1 Locally and Test - Hands-on Tutorial
This video shows how to locally install Meta Llama 3.1 8B model and test it on various benchmarks.
Code:
conda create -n newllama python=3.11 -y && conda activate newllama
pip install torch
pip install --upgrade transformers
pip install accelerate huggingface_hub
huggingface-cli login # get the key from huggingface.co
conda install jupyter -y
pip uninstall charset_normalizer -y
pip install charset_normalizer
jupyter notebook
import transformers
import torch
model_id = "meta-llama/Meta-Llama-3.1-8B-Instruct" # make sure to accept the terms on model card.
pipeline = transformers.pipeline(
"text-generation",
model=model_id,
model_kwargs={"torch_dtype": torch.bfloat16},
device_map="auto",
)
messages = [
{"role": "system", "content": "You are helpful assistant!"},
{"role": "user", "content": "What is the smallest country in the world?"},
]
outputs = pipeline(
messages,
max_new_tokens=256,
)
from IPython.display import Markdown, display
output_text = outputs[0]["generated_text"][-1]['content']
display(Markdown(output_text))
Categories: DBA Blogs
Llama 3 Groq 70B Tool Use Model - Local Installation and Function Calling
This video installs Llama-3-Groq-8B-Tool-Use locally which is specifically designed for advanced tool use and function calling tasks.
Code:
conda create -n groqllama python=3.11 -y && conda activate groqllama
conda install jupyter -y
pip uninstall charset_normalizer -y
pip install charset_normalizer
jupyter notebook
pip install torch transformers sentencepiece accelerate huggingface_hub tavily-python
export TAVILY_API_KEY=""
import transformers
import torch
import os
import re
import json
from tavily import TavilyClient
tavily_client = TavilyClient(api_key=os.getenv('TAVILY_API_KEY'))
import warnings
warnings.filterwarnings('ignore')
model_id = "Groq/Llama-3-Groq-8B-Tool-Use"
pipeline = transformers.pipeline(
"text-generation",
model=model_id,
model_kwargs={"torch_dtype": torch.bfloat16},
device_map="auto",
)
prompt ="""
<|start_header_id|>system<|end_header_id|>
You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:
<tool_call>
{"name": <function-name>,"arguments": <args-dict>}
</tool_call>
Here are the available tools:
<tools> {
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
},
"format": {
"type": "string",
"description": "The temperature unit to use. Infer this from the users location.",
"enum": [
"celsius",
"fahrenheit"
]
}
},
"required": [
"location",
"format"
]
}
} </tools><|eot_id|><|start_header_id|>user<|end_header_id|>
What is the weather like in Sydney in Celsius?<|eot_id|><|start_header_id|>assistant<|end_header_id|>
"""
response = pipeline(prompt)
# Use a regex pattern to find the tool call JSON
generated_text = response[0].get('generated_text', '')
# Use a regex pattern to find the tool call JSON
tool_call_match = re.search(r'\{.*?\}', generated_text, re.DOTALL)
if tool_call_match:
tool_call_json = tool_call_match.group(0)
# Correctly format the JSON string
tool_call_json = tool_call_json.replace("<function-name>", "get_current_weather") # Replace placeholder
tool_call_json = tool_call_json.replace("<args-dict>", '{"location": "Sydney, NSW", "format": "celsius"}') # Replace placeholder
tool_call_json = tool_call_json.replace("'", '"') # Replace single quotes with double quotes
tool_call_json = tool_call_json.replace('name:', '"name":') # Ensure proper quoting for keys
tool_call_json = tool_call_json.replace('arguments:', '"arguments":')
# Ensure proper quoting of all parts of the JSON string
tool_call_json = tool_call_json.replace('"name": get_current_weather', '"name": "get_current_weather"')
# Debug: Print the extracted JSON string
#print(f"Extracted JSON: {tool_call_json}")
# Correctly format the JSON string
try:
tool_call = json.loads(tool_call_json)
print(tool_call)
except json.JSONDecodeError as e:
print(f"Error decoding JSON: {e}")
# Debug: Print the exact content that failed to parse
print(f"Failed JSON content: {tool_call_json}")
else:
print("No tool call JSON found.")
location=tool_call['arguments']['location']
format_unit=tool_call['arguments']['format']
query = f"current weather in {location} in {format_unit}"
response = tavily_client.search(query)
print(response)
Categories: DBA Blogs
Create Your Own Planner with GPT4o Mini Locally
This video is a step-by-step easy tutorial to create a generic planner with API calls and Gradio interface by using GPT4o Mini.
Code:
#pip install openai gradio
#export OPENAI_API_KEY=""
import openai
import os
import gradio as gr
client = openai.OpenAI(api_key=os.environ.get('OPENAI_API_KEY'))
def generate_plans(user_query, n=5):
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "Plan and respond to the user query."},
{"role": "user", "content": user_query}
],
n=n,
temperature=0.7,
max_tokens=500,
stop=['']
)
plans = [choice.message.content for choice in response.choices if choice.message.content.strip() != '']
if not plans:
plans = ["Plan A", "Plan B", "Plan C"]
return plans
def compare_plans(plan1, plan2):
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "Choose the better plan."},
{"role": "user", "content": f"Plan 1: {plan1}\n\nPlan 2: {plan2}\n\nWhich plan is better? Respond with either '1' or '2'."}
],
temperature=0.2,
max_tokens=10
)
return response.choices[0].message.content.strip() if response.choices[0].message.content.strip() != '' else '1'
def evaluate_plans(plans, user_query):
winners = plans
while len(winners) > 1:
next_round = []
for i in range(0, len(winners), 2):
if i+1 < len(winners):
winner = winners[i] if compare_plans(winners[i], winners[i+1]) == '1' else winners[i+1]
else:
winner = winners[i]
next_round.append(winner)
winners = next_round
return winners[0] if winners else 'No best plan found'
def generate_response(best_plan, user_query):
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "Respond to the user query based on the plan."},
{"role": "user", "content": f"User Query: {user_query}\n\nPlan: {best_plan}\n\nGenerate a detailed response."}
],
temperature=0.5,
max_tokens=2000
)
return response.choices[0].message.content
def improved_ai_output(user_query, num_plans=20):
plans = generate_plans(user_query, n=num_plans)
best_plan = evaluate_plans(plans, user_query)
final_response = generate_response(best_plan, user_query)
return {
"user_query": user_query,
"best_plan": best_plan,
"final_response": final_response
}
def chat(query):
result = improved_ai_output(query)
return result['final_response']
interface = gr.Interface(
fn=chat,
inputs=gr.Textbox(lines=2, placeholder="Ask me anything..."),
outputs=gr.Textbox(),
title="My Planner",
description="Get a personalized plan as per your requirement!"
)
if __name__ == "__main__":
interface.launch()
Categories: DBA Blogs
RAG Pipeline Tutorial Using Ollama, Triplex, and LangChain On Custom Data Locally
This video is a step-by-step guide on building an end-to-end RAG pipeline on your own custom data locally by using Ollama models Triplex and Langchain with GUI in Gradio.
Code:
conda create -n ragpipe python=3.11 -y && conda activate ragpipe
pip install torch sentence_transformers transformers accelerate
pip install langchain==0.1.14
pip install langchain-experimental==0.0.56
pip install langchain-community==0.0.31
pip install faiss-cpu==1.8.0
pip install pdfplumber==0.11.0
pip install gradio==4.25.0
pip install ollama
pip install pypdf
conda install jupyter -y
pip uninstall charset_normalizer -y
pip install charset_normalizer
jupyter notebook
from langchain_community.document_loaders import PDFPlumberLoader
from langchain_experimental.text_splitter import SemanticChunker
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_community.llms import Ollama
from langchain.prompts import PromptTemplate
from langchain.chains.llm import LLMChain
from langchain.chains.combine_documents.stuff import StuffDocumentsChain
from langchain.chains import RetrievalQA
from langchain_community.document_loaders import TextLoader
from pypdf import PdfReader
import ollama
import gradio as gr
import os
import json
def triplextract(text, entity_types, predicates):
input_format = """
**Entity Types:**
{entity_types}
**Predicates:**
{predicates}
**Text:**
{text}
"""
message = input_format.format(
entity_types = json.dumps({"entity_types": entity_types}),
predicates = json.dumps({"predicates": predicates}),
text = text)
# Pass the message as a single string
prompt = message
output = ollama.generate(model='triplex', prompt=prompt)
return output
entity_types = ["PERSON", "LOCATION"]
predicates = ["PROFESSION", "BASED_IN"]
reader = PdfReader("/home/Ubuntu/myfiles/mypdf.pdf")
text = ""
for page in reader.pages:
text += page.extract_text() + "\n"
prediction = triplextract(text, entity_types, predicates)
response_string = prediction['response'].strip('```json\n').strip()
response_string = response_string.lstrip('\n')
response_string = response_string.strip('```')
response_string = response_string.replace('```', '')
response_string = response_string.replace("json", "")
response_json = json.loads(response_string)
entities_and_triples = response_json['entities_and_triples']
print(entities_and_triples)
with open('output.txt', 'w') as f:
f.write(text)
loader = TextLoader("./output.txt")
docs =loader.load()
# Split into chunks
text_splitter = SemanticChunker(HuggingFaceEmbeddings())
documents = text_splitter.split_documents(docs)
# Instantiate the embedding model
embedder = HuggingFaceEmbeddings()
# Create the vector store and fill it with embeddings
vector = FAISS.from_documents(documents, embedder)
retriever = vector.as_retriever(search_type="similarity", search_kwargs={"k": 3})
# Define llm
llm = Ollama(model="mistral")
# Define the prompt
prompt = """
1. Use the following pieces of context to answer the question at the end.
2. If you don't know the answer, just say that "I don't know" but don't make up an answer on your own.\n
3. Keep the answer crisp and limited to 3,4 sentences.
Context: {context}
Question: {question}
Helpful Answer:"""
QA_CHAIN_PROMPT = PromptTemplate.from_template(prompt)
llm_chain = LLMChain(
llm=llm,
prompt=QA_CHAIN_PROMPT,
callbacks=None,
verbose=True)
document_prompt = PromptTemplate(
input_variables=["page_content", "source"],
template="Context:\ncontent:{page_content}\nsource:{source}",
)
combine_documents_chain = StuffDocumentsChain(
llm_chain=llm_chain,
document_variable_name="context",
document_prompt=document_prompt,
callbacks=None)
qa = RetrievalQA(
combine_documents_chain=combine_documents_chain,
verbose=True,
retriever=retriever,
return_source_documents=True)
def respond(question,history):
return qa(question)["result"]
gr.ChatInterface(
respond,
chatbot=gr.Chatbot(height=500),
textbox=gr.Textbox(placeholder="Ask me question related to Fahd Mirza", container=False, scale=7),
title="Fahd's Chatbot",
examples=["Where Fahd Lives", "Who is Fahd"],
cache_examples=True,
retry_btn=None,
).launch(share = True)
Categories: DBA Blogs
GraphRAG Replacement - SciPhi Triplex - Step by Step Local Installation
This video installs Triplex which is a finetuned version of Phi3-3.8B for creating knowledge graphs from unstructured data developed by SciPhi.AI. It works by extracting triplets.
Code:
conda create -n triplex python=3.11 -y && conda activate triplex
pip install torch transformers accelerate
import json
from transformers import AutoModelForCausalLM, AutoTokenizer
def triplextract(model, tokenizer, text, entity_types, predicates):
input_format = """
**Entity Types:**
{entity_types}
**Predicates:**
{predicates}
**Text:**
{text}
"""
message = input_format.format(
entity_types = json.dumps({"entity_types": entity_types}),
predicates = json.dumps({"predicates": predicates}),
text = text)
messages = [{'role': 'user', 'content': message}]
input_ids = tokenizer.apply_chat_template(messages, add_generation_prompt = True, return_tensors="pt").to("cuda")
output = tokenizer.decode(model.generate(input_ids=input_ids, max_length=2048)[0], skip_special_tokens=True)
return output
model = AutoModelForCausalLM.from_pretrained("sciphi/triplex", trust_remote_code=True).to('cuda').eval()
tokenizer = AutoTokenizer.from_pretrained("sciphi/triplex", trust_remote_code=True)
entity_types = [ "LOCATION", "POSITION", "DATE", "CITY", "COUNTRY", "NUMBER" ]
predicates = [ "POPULATION", "AREA" ]
text = """
San Francisco,[24] officially the City and County of San Francisco, is a commercial, financial, and cultural center in Northern California.
With a population of 808,437 residents as of 2022, San Francisco is the fourth most populous city in the U.S. state of California behind Los Angeles, San Diego, and San Jose.
"""
prediction = triplextract(model, tokenizer, text, entity_types, predicates)
print(prediction)
entity_types = ["CASE", "LAWYER", "DATE"]
predicates = ["VERDICT", "CHARGES"]
text = """
In the landmark case of Roe v. Wade (1973), lawyer Sarah Weddington successfully argued before the US Supreme Court, leading to a verdict that protected women's reproductive rights.
"""
prediction = triplextract(model, tokenizer, text, entity_types, predicates)
print(prediction)
Categories: DBA Blogs
Create Space Shooter Game with Local Free AI Model Mistral Nemo
This video show how to create a free local space shooter game with just text prompt using Mistral Nemo LLM.
Full Prompt:
"""
Create a Python program that generates a customizable space shooter game. The game should have the following features:
A player ship that can be customized with a choice of 3 colors and 2 weapon types
3 types of enemies with unique behaviors and difficulty levels
2 power-ups that can be collected by the player
5 levels with increasing difficulty
A boss fight at the end of level 5
The program should use the Pygame library and generate the game assets (sprites, backgrounds, etc.) using AI algorithms. The game should be playable from the terminal.
Customization Options:
Player ship color (red, blue, or green)
Player ship weapon type (laser or missiles)
AI-Generated Assets:
Player ship sprite
Enemy sprites (3 types)
Power-up icons (2 types)
Backgrounds (5 levels)
Gameplay Logic:
Player movement and shooting
Enemy AI behaviors (patrolling, chasing, etc.)
Collision detection and response
Scoring and level progression
Test the AI model's ability to generate a functional and engaging space shooter game with customization options.
"""
Full Game Code:
```python
import pygame
import random
# Initialize Pygame
pygame.init()
# Set up some constants
WIDTH, HEIGHT = 800, 600
FPS = 60
WHITE = (255, 255, 255)
BLACK = (0, 0, 0)
RED = (255, 0, 0)
LIVES = 3
# Set up the display
screen = pygame.display.set_mode((WIDTH, HEIGHT))
pygame.display.set_caption("Space Shooter")
clock = pygame.time.Clock()
# Player class
class Player(pygame.sprite.Sprite):
def __init__(self):
super().__init__()
self.image = pygame.Surface((50, 50))
self.image.fill(WHITE)
self.rect = self.image.get_rect()
self.rect.centerx = WIDTH // 2
self.rect.bottom = HEIGHT - 10
self.speed_x = 0
def update(self):
self.speed_x = 0
keys = pygame.key.get_pressed()
if keys[pygame.K_LEFT]:
print("Left key pressed")
self.speed_x = -5
if keys[pygame.K_RIGHT]:
print("Right key pressed")
self.speed_x = 5
self.rect.x += self.speed_x
if self.rect.left < 0:
self.rect.left = 0
if self.rect.right > WIDTH:
self.rect.right = WIDTH
def shoot(self):
bullet = Bullet(self.rect.centerx, self.rect.top)
all_sprites.add(bullet)
bullets.add(bullet)
# Enemy class
class Enemy(pygame.sprite.Sprite):
def __init__(self):
super().__init__()
self.image = pygame.Surface((30, 30))
self.image.fill(RED)
self.rect = self.image.get_rect()
self.rect.x = random.randrange(WIDTH - self.rect.width)
self.rect.y = random.randrange(-100, -40)
self.speed_y = random.randrange(1, 8)
def update(self):
self.rect.y += self.speed_y
if self.rect.top > HEIGHT + 10:
self.rect.x = random.randrange(WIDTH - self.rect.width)
self.rect.y = random.randrange(-100, -40)
self.speed_y = random.randrange(1, 8)
# Bullet class
class Bullet(pygame.sprite.Sprite):
def __init__(self, x, y):
super().__init__()
self.image = pygame.Surface((10, 20))
self.image.fill(WHITE)
self.rect = self.image.get_rect()
self.rect.centerx = x
self.rect.top = y
self.speed_y = -10
def update(self):
self.rect.y += self.speed_y
if self.rect.bottom < 0:
self.kill()
# Create sprite groups
all_sprites = pygame.sprite.Group()
enemies = pygame.sprite.Group()
bullets = pygame.sprite.Group()
# Create player instance
player = Player()
all_sprites.add(player)
# Create enemy instances
for i in range(10):
enemy = Enemy()
all_sprites.add(enemy)
enemies.add(enemy)
# Main game loop
running = True
lives = LIVES
while running:
clock.tick(FPS)
# Process input (events)
for event in pygame.event.get():
if event.type == pygame.QUIT:
running = False
elif event.type == pygame.KEYDOWN:
if event.key == pygame.K_SPACE:
print("Spacebar pressed")
player.shoot()
# Update
all_sprites.update()
# Check for bullet-enemy collisions
hits = pygame.sprite.groupcollide(enemies, bullets, True, True)
for hit in hits:
enemy = Enemy()
all_sprites.add(enemy)
enemies.add(enemy)
# Check if player collides with an enemy
if pygame.sprite.spritecollideany(player, enemies):
lives -= 1
print(f"Lives left: {lives}")
if lives == 0:
running = False
else:
player.rect.centerx = WIDTH // 2
player.rect.bottom = HEIGHT - 10
# Draw
screen.fill(BLACK)
all_sprites.draw(screen)
# Flip the display
pygame.display.flip()
pygame.quit()
Categories: DBA Blogs
Use GPT-4o Mini Locally with Text and Images
This video introduces and shows how to use GPT-4o mini by OpenAI which is quite cost efficient and performant.
Code:
from openai import OpenAI
import base64
import requests
import os
## Set the API key and model name
MODEL="gpt-4o-mini"
os.environ.get('OPENAI_API_KEY')
client = OpenAI(api_key=os.environ.get('OPENAI_API_KEY'))
def encode_image(image_path):
with open(image_path, "rb") as image_file:
return base64.b64encode(image_file.read()).decode("utf-8")
IMAGE_PATH="nm.png"
base64_image = encode_image(IMAGE_PATH)
response = client.chat.completions.create(
model=MODEL,
messages=[
{"role": "system", "content": "You are a helpful assistant that responds in Markdown. Help me with this image!"},
{"role": "user", "content": [
{"type": "text", "text": "Describe the image? how many girls are there?"},
{"type": "image_url", "image_url": {
"url": f"data:image/png;base64,{base64_image}"}
}
]}
],
temperature=0.0,
)
print(response.choices[0].message.content)
-
#pip install -U openai
#export OPENAI_API_KEY=""
from openai import OpenAI
import os
## Set the API key and model name
MODEL="gpt-4o-mini"
os.environ.get('OPENAI_API_KEY')
client = OpenAI(api_key=os.environ.get('OPENAI_API_KEY'))
completion = client.chat.completions.create(
model=MODEL,
messages=[
{"role": "system", "content": "You are a helpful assistant. Help me with my question!"},
{"role": "user", "content": "A bat and a ball together cost $1.10. The bat costs $1.00 more than the ball. How much does the ball cost?"}
]
)
print("Assistant: " + completion.choices[0].message.content)
Categories: DBA Blogs
Install Mistral Nemo Locally and Test for Multi-Lingual , Function Calling
This video installs Mistral NeMo locally and tests it on multi-lingual, math, coding, and function calling.
Code:
conda create -n nemo python=3.11 -y && conda activate nemo
pip install torch
pip install git+https://github.com/huggingface/transformers.git
pip install mistral_inference
pip install huggingface_hub pathlib
from huggingface_hub import snapshot_download
from pathlib import Path
mistral_models_path = Path.home().joinpath('mistral_models', 'Nemo-Instruct')
mistral_models_path.mkdir(parents=True, exist_ok=True)
snapshot_download(repo_id="mistralai/Mistral-Nemo-Instruct-2407", allow_patterns=["params.json", "consolidated.safetensors", "tekken.json"], local_dir=mistral_models_path)
conda install jupyter -y
pip uninstall charset_normalizer -y
pip install charset_normalizer
jupyter notebook
from mistral_inference.transformer import Transformer
from mistral_inference.generate import generate
from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
from mistral_common.protocol.instruct.messages import UserMessage
from mistral_common.protocol.instruct.request import ChatCompletionRequest
tokenizer = MistralTokenizer.from_file(f"{mistral_models_path}/tekken.json")
model = Transformer.from_folder(mistral_models_path)
prompt = "Write 10 sentences ending with the word beauty."
completion_request = ChatCompletionRequest(messages=[UserMessage(content=prompt)])
tokens = tokenizer.encode_chat_completion(completion_request).tokens
out_tokens, _ = generate([tokens], model, max_tokens=64, temperature=0.35, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)
result = tokenizer.decode(out_tokens[0])
print(result)
#===============================
# Function Calling
#===============================
from mistral_common.protocol.instruct.tool_calls import Function, Tool
from mistral_inference.transformer import Transformer
from mistral_inference.generate import generate
from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
from mistral_common.protocol.instruct.messages import UserMessage
from mistral_common.protocol.instruct.request import ChatCompletionRequest
completion_request = ChatCompletionRequest(
tools=[
Tool(
function=Function(
name="get_current_weather",
description="Get the current weather",
parameters={
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"format": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to use. Infer this from the users location.",
},
},
"required": ["location", "format"],
},
)
)
],
messages=[
UserMessage(content="What's the weather like today in Paris?"),
],
)
tokens = tokenizer.encode_chat_completion(completion_request).tokens
out_tokens, _ = generate([tokens], model, max_tokens=256, temperature=0.35, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)
result = tokenizer.decode(out_tokens[0])
print(result)
Categories: DBA Blogs
Install Codestral Mamba Locally - Best Math AI Model
This video installs Codestral Mamba locally which is an open code model based on the Mamba2 architecture.
Code:
conda create -n codestralmamba python=3.11 -y && conda activate codestralmamba
pip install torch huggingface_hub pathlib2
pip install mistral_inference>=1 mamba-ssm causal-conv1d
from huggingface_hub import snapshot_download
from pathlib import Path
mistral_models_path = Path.home().joinpath('mistral_models', 'mamba-codestral-7B-v0.1')
mistral_models_path.mkdir(parents=True, exist_ok=True)
snapshot_download(repo_id="mistralai/mamba-codestral-7B-v0.1", allow_patterns=["params.json", "consolidated.safetensors", "tokenizer.model.v3"], local_dir=mistral_models_path)
mistral-chat $HOME/mistral_models/mamba-codestral-7B-v0.1 --instruct --max_tokens 256
Categories: DBA Blogs
Easy Tutorial to Fine-Tune Vision Model on Image Data Locally
This video is a step-by-step easy tutorial to fine-tune any vision model on your own custom image dataset locally easily.
Code:
conda create -n ft python=3.11 -y && conda activate ft
!pip install -U transformers datasets trl peft accelerate Pillow torch
from datasets import features, load_dataset
from transformers import AutoModelForVision2Seq, AutoProcessor
import torch
from trl import DPOConfig, DPOTrainer
from peft import LoraConfig
import os
ds_id = "openbmb/RLAIF-V-Dataset"
dataset = load_dataset(ds_id, split="train")
dataset = dataset.shuffle(seed=42).select(range(100))
model_id = "HuggingFaceM4/idefics2-8b"
model = AutoModelForVision2Seq.from_pretrained(model_id, torch_dtype=torch.bfloat16)
processor = AutoProcessor.from_pretrained(model_id, do_image_splitting=False)
def format_ds(example):
# Prepare the input for the chat template
prompt = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": example["question"]}]}]
chosen = [{"role": "assistant", "content": [{"type": "text", "text": example["chosen"]}]}]
rejected = [{"role": "assistant", "content": [{"type": "text", "text": example["rejected"]}]}]
# Apply the chat template
prompt = processor.apply_chat_template(prompt, tokenize=False)
chosen = processor.apply_chat_template(chosen, tokenize=False)
rejected = processor.apply_chat_template(rejected, tokenize=False)
# Resize the image to ensure it fits within the maximum allowable
# size of the processor to prevent OOM errors.
max_size = processor.image_processor.size["longest_edge"] // 2
example["image"].thumbnail((max_size, max_size))
return {"images": [example["image"]], "prompt": prompt, "chosen": chosen, "rejected": rejected}
dataset = dataset.map(format_ds, remove_columns=dataset.column_names, num_proc=os.cpu_count())
f = dataset.features
f["images"] = features.Sequence(features.Image(decode=True))
dataset = dataset.cast(f)
training_args = DPOConfig(
output_dir="my-idefics2",
bf16=True,
gradient_checkpointing=True,
per_device_train_batch_size=2,
gradient_accumulation_steps=32,
num_train_epochs=1,
push_to_hub=False,
dataset_num_proc=os.cpu_count(),
dataloader_num_workers=os.cpu_count(),
logging_steps=10,
)
trainer = DPOTrainer(
model,
ref_model=None, # not needed when using peft
args=training_args,
train_dataset=dataset,
tokenizer=processor,
peft_config=LoraConfig(target_modules="all-linear"),
)
trainer.train()
Categories: DBA Blogs
Install aTrain Locally - Offline Transcription of Speech Recordings Free with AI
This video is a step-by-step easy tutorial to install aTrain which is a tool for automatically transcribing speech recordings utilizing state-of-the-art machine learning models without uploading any data.
Code:
conda create -n atrain python=3.11 -y && conda activate atrain
sudo apt install ffmpeg
pip install aTrain@git+https://github.com/BANDAS-Center/aTrain.git --extra-index-url https://download.pytorch.org/whl/cu118
sudo apt install nvidia-cudnn
aTrain init
aTrain start
Categories: DBA Blogs
llama-cpp-agent Installation to Use AI Models Locally in Simple Way
This video is a step-by-step easy tutorial to install llama-cpp-agent which is a tool designed to simplify interactions with LLMs. It provides an interface for chatting with LLMs, executing function calls, generating structured output.
Code:
conda create -n llamagent python=3.11 -y && conda activate llamagent
pip install --no-cache-dir llama-cpp-python==0.2.77 --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu124
pip install llama-cpp-agent
conda install jupyter -y
pip uninstall charset_normalizer -y
pip install charset_normalizer
jupyter notebook
# Import the Llama class of llama-cpp-python and the LlamaCppPythonProvider of llama-cpp-agent
from llama_cpp import Llama
from llama_cpp_agent.providers import LlamaCppPythonProvider
# Create an instance of the Llama class and load the model
llama_model = Llama(r"/home/Ubuntu/mymodels/mistral-7b-instruct-v0.2.Q6_K.gguf", n_batch=1024, n_threads=10, n_gpu_layers=40)
# Create the provider by passing the Llama class instance to the LlamaCppPythonProvider class
provider = LlamaCppPythonProvider(llama_model)
from llama_cpp_agent import LlamaCppAgent
from llama_cpp_agent import MessagesFormatterType
agent = LlamaCppAgent(provider, system_prompt="You are a helpful assistant.", predefined_messages_formatter_type=MessagesFormatterType.MISTRAL)
agent_output = agent.get_chat_response("Hello, World!")
print(f"Agent: {agent_output.strip()}")
import math
from llama_cpp_agent import FunctionCallingAgent
from llama_cpp_agent.llm_output_settings import LlmStructuredOutputSettings
from typing import Union
# Callback for receiving messages for the user.
def send_message_to_user_callback(message: str):
print(message)
def calculate_a_to_the_power_b(a: Union[int, float], b: Union[int, float]):
"""
Calculates a to the power of b
Args:
a: number
b: exponent
"""
return f"Result: {math.pow(a, b)}"
output_settings = LlmStructuredOutputSettings.from_functions([calculate_a_to_the_power_b], allow_parallel_function_calling=True)
llama_cpp_agent = LlamaCppAgent(
provider,
debug_output=True,
system_prompt=f"You are an advanced AI, tasked to assist the user by calling functions in JSON format.",
predefined_messages_formatter_type=MessagesFormatterType.CHATML,
)
user_input = "Calculate a to the power of b: a = 2, b = 3"
print(
llama_cpp_agent.get_chat_response(
user_input, structured_output_settings=output_settings
)
)
Categories: DBA Blogs
AuraFlow - Best and Free Text to Image Model - Install Locally
This video installs AuraFlow v0.1 locally which is the fully open-sourced largest flow-based text-to-image generation model. It generates hyper realistic images from text prompts.
Code:
conda create -n auraflow python=3.11 -y && conda activate auraflow
pip install transformers accelerate protobuf sentencepiece
pip install torch torchvision
pip install git+https://github.com/huggingface/diffusers.git@bbd2f9d4e9ae70b04fedf65903fd1fb035437db4
conda install jupyter -y
pip uninstall charset_normalizer -y
pip install charset_normalizer
jupyter notebook
from diffusers import AuraFlowPipeline
import torch
pipeline = AuraFlowPipeline.from_pretrained(
"fal/AuraFlow",
torch_dtype=torch.float16
).to("cuda")
image = pipeline(
prompt="close-up portrait of a majestic iguana with vibrant blue-green scales, piercing amber eyes, and orange spiky crest. Intricate textures and details visible on scaly skin. Wrapped in dark hood, giving regal appearance. Dramatic lighting against black background. Hyper-realistic, high-resolution image showcasing the reptile's expressive features and coloration.",
height=1024,
width=1024,
num_inference_steps=50,
generator=torch.Generator().manual_seed(666),
guidance_scale=3.5,
).images[0]
Categories: DBA Blogs
Install LightRAG Locally - Moduler RAG and Lightning Library for LLM Apps
This video is a step-by-step easy tutorial to install LightRAG which helps developers with both building and optimizing Retriever-Agent-Generator pipelines. It is light, modular, and robust, with a 100% readable codebase.
Code:
conda create -n lightrag python=3.11 -y && conda activate lightrag
git clone https://github.com/SylphAI-Inc/LightRAG.git && cd LightRAG
pip install lightrag
pip install openai==1.12.0
pip install faiss-cpu==1.8.0
pip install sqlalchemy==2.0.30
pip install pgvector==0.2.5
pip install groq==0.5.0
mv .env_example .env #set your openai and groq api keys in .env file.
conda install jupyter -y
pip uninstall charset_normalizer -y
pip install charset_normalizer
jupyter notebook
from lightrag.utils import setup_env
setup_env()
from dataclasses import dataclass, field
from lightrag.core import Component, Generator, DataClass
from lightrag.components.model_client import GroqAPIClient
from lightrag.components.output_parsers import JsonOutputParser
@dataclass
class QAOutput(DataClass):
explanation: str = field(
metadata={"desc": "A brief explanation of the concept in one sentence."}
)
example: str = field(metadata={"desc": "An example of the concept in a sentence."})
qa_template = r"""<SYS>
You are a helpful assistant.
<OUTPUT_FORMAT>
{{output_format_str}}
</OUTPUT_FORMAT>
</SYS>
User: {{input_str}}
You:"""
class QA(Component):
def __init__(self):
super().__init__()
parser = JsonOutputParser(data_class=QAOutput, return_data_class=True)
self.generator = Generator(
model_client=GroqAPIClient(),
model_kwargs={"model": "llama3-8b-8192"},
template=qa_template,
prompt_kwargs={"output_format_str": parser.format_instructions()},
output_processors=parser,
)
def call(self, query: str):
return self.generator.call({"input_str": query})
async def acall(self, query: str):
return await self.generator.acall({"input_str": query})
qa = QA()
print(qa)
# call
output = qa("What is LLM?")
print(output)
qa.generator.print_prompt(
output_format_str=qa.generator.output_processors.format_instructions(),
input_str="What is LLM?",
)
Categories: DBA Blogs
Microsoft GraphRAG Installation with Llama.CPP Locally - Step-by-Step Guide
This video is a step-by-step tutorial to install Microsoft GraphRAG with Llama.cpp and Groq API models with your own data.
Code:
conda create -n graphcpp python=3.11 -y && conda activate graphcpp
./llama-server -m /home/Ubuntu/mymodels/qwen1_5-4b-chat-q4_k_m.gguf -c 8192 -n -1 -t 7 --embeddings
pip install graphrag
mkdir -p ./ragtest/input
cp fahd.txt ragtest/input
python3 -m graphrag.index --init --root ./ragtest
python3 -m graphrag.index --root ./ragtest
python3 -m graphrag.query --root ./ragtest --method global "Who is Fahd Mirza?"
python3 -m graphrag.query --root ./ragtest --method local "Who is Fahd Mirza and where does he live?"
Settings.yaml
encoding_model: cl100k_base
skip_workflows: []
llm:
api_key: ${GRAPHRAG_API_KEY}
type: openai_chat # or azure_openai_chat
model: mixtral-8x7b-32768
model_supports_json: false # recommended if this is available for your model.
api_base: https://api.groq.com/openai/v1
max_tokens: 8192
concurrent_requests: 1 # the number of parallel inflight requests that may be made
tokens_per_minute: 28000 # set a leaky bucket throttle
requests_per_minute: 29 # set a leaky bucket throttle
# request_timeout: 180.0
# api_version: 2024-02-15-preview
# organization: <organization_id>
# deployment_name: <azure_model_deployment_name>
max_retries: 10
max_retry_wait: 60.0
sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
parallelization:
stagger: 0.3
# num_threads: 50 # the number of threads to use for parallel processing
async_mode: threaded # or asyncio
embeddings:
## parallelization: override the global parallelization settings for embeddings
async_mode: threaded # or asyncio
llm:
api_key: ${GRAPHRAG_API_KEY}
type: openai_embedding # or azure_openai_embedding
model: text-embedding-ada-002
api_base: http://localhost:8080
batch_size: 1 # the number of documents to send in a single request
# api_version: 2024-02-15-preview
# organization: <organization_id>
# deployment_name: <azure_model_deployment_name>
# tokens_per_minute: 150_000 # set a leaky bucket throttle
# requests_per_minute: 10_000 # set a leaky bucket throttle
# max_retries: 10
# max_retry_wait: 10.0
# sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
# concurrent_requests: 25 # the number of parallel inflight requests that may be made
# batch_max_tokens: 8191 # the maximum number of tokens to send in a single request
# target: required # or optional
chunks:
size: 300
overlap: 100
group_by_columns: [id] # by default, we don't allow chunks to cross documents
input:
type: file # or blob
file_type: text # or csv
base_dir: "input"
file_encoding: utf-8
file_pattern: ".*\\.txt$"
cache:
type: file # or blob
base_dir: "cache"
# connection_string: <azure_blob_storage_connection_string>
# container_name: <azure_blob_storage_container_name>
storage:
type: file # or blob
base_dir: "output/${timestamp}/artifacts"
# connection_string: <azure_blob_storage_connection_string>
# container_name: <azure_blob_storage_container_name>
reporting:
type: file # or console, blob
base_dir: "output/${timestamp}/reports"
# connection_string: <azure_blob_storage_connection_string>
# container_name: <azure_blob_storage_container_name>
entity_extraction:
## llm: override the global llm settings for this task
## parallelization: override the global parallelization settings for this task
## async_mode: override the global async_mode settings for this task
prompt: "prompts/entity_extraction.txt"
entity_types: [organization,person,geo,event]
max_gleanings: 0
summarize_descriptions:
## llm: override the global llm settings for this task
## parallelization: override the global parallelization settings for this task
## async_mode: override the global async_mode settings for this task
prompt: "prompts/summarize_descriptions.txt"
max_length: 500
claim_extraction:
## llm: override the global llm settings for this task
## parallelization: override the global parallelization settings for this task
## async_mode: override the global async_mode settings for this task
# enabled: true
prompt: "prompts/claim_extraction.txt"
description: "Any claims or facts that could be relevant to information discovery."
max_gleanings: 0
community_report:
## llm: override the global llm settings for this task
## parallelization: override the global parallelization settings for this task
## async_mode: override the global async_mode settings for this task
prompt: "prompts/community_report.txt"
max_length: 2000
max_input_length: 8000
cluster_graph:
max_cluster_size: 10
embed_graph:
enabled: false # if true, will generate node2vec embeddings for nodes
# num_walks: 10
# walk_length: 40
# window_size: 2
# iterations: 3
# random_seed: 597832
umap:
enabled: false # if true, will generate UMAP embeddings for nodes
snapshots:
graphml: false
raw_entities: false
top_level_nodes: false
local_search:
# text_unit_prop: 0.5
# community_prop: 0.1
# conversation_history_max_turns: 5
# top_k_mapped_entities: 10
# top_k_relationships: 10
# max_tokens: 12000
global_search:
# max_tokens: 12000
# data_max_tokens: 12000
# map_max_tokens: 1000
# reduce_max_tokens: 2000
# concurrency: 32
Categories: DBA Blogs
Install Microsoft GraphRAG with Groq API Locally
This video is a step-by-step tutorial to install Microsoft GraphRAG with Groq API models with your own data.
Code:
conda create -n graphgroq python=3.11 -y && conda activate graphgroq
export OPENAI_API_KEY=""
export GROQ_API_KEY=""
pip install graphrag
mkdir -p ./ragtest/input
cp fahd.txt ragtest/input
python3 -m graphrag.index --init --root ./ragtest
python3 -m graphrag.index --root ./ragtest
python3 -m graphrag.query --root ./ragtest --method global "Who is Fahd Mirza?"
python3 -m graphrag.query --root ./ragtest --method local "Who is Fahd Mirza and where does he live?"
Settings.yaml File:
encoding_model: cl100k_base
skip_workflows: []
llm:
api_key: ${GRAPHRAG_API_KEY}
type: openai_chat # or azure_openai_chat
model: mixtral-8x7b-32768
model_supports_json: false # recommended if this is available for your model.
api_base: https://api.groq.com/openai/v1
max_tokens: 8192
concurrent_requests: 1 # the number of parallel inflight requests that may be made
tokens_per_minute: 28000 # set a leaky bucket throttle
requests_per_minute: 29 # set a leaky bucket throttle
# request_timeout: 180.0
# api_version: 2024-02-15-preview
# organization: <organization_id>
# deployment_name: <azure_model_deployment_name>
max_retries: 10
max_retry_wait: 60.0
sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
parallelization:
stagger: 0.3
# num_threads: 50 # the number of threads to use for parallel processing
async_mode: threaded # or asyncio
embeddings:
## parallelization: override the global parallelization settings for embeddings
async_mode: threaded # or asyncio
llm:
api_key: ${OPENAI_API_KEY}
type: openai_embedding # or azure_openai_embedding
model: text-embedding-3-small
api_base: https://api.openai.com/v1
# api_base: https://<instance>.openai.azure.com
# api_version: 2024-02-15-preview
# organization: <organization_id>
# deployment_name: <azure_model_deployment_name>
# tokens_per_minute: 150_000 # set a leaky bucket throttle
# requests_per_minute: 10_000 # set a leaky bucket throttle
# max_retries: 10
# max_retry_wait: 10.0
# sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
# concurrent_requests: 25 # the number of parallel inflight requests that may be made
# batch_size: 16 # the number of documents to send in a single request
# batch_max_tokens: 8191 # the maximum number of tokens to send in a single request
# target: required # or optional
chunks:
size: 300
overlap: 100
group_by_columns: [id] # by default, we don't allow chunks to cross documents
input:
type: file # or blob
file_type: text # or csv
base_dir: "input"
file_encoding: utf-8
file_pattern: ".*\\.txt$"
cache:
type: file # or blob
base_dir: "cache"
# connection_string: <azure_blob_storage_connection_string>
# container_name: <azure_blob_storage_container_name>
storage:
type: file # or blob
base_dir: "output/${timestamp}/artifacts"
# connection_string: <azure_blob_storage_connection_string>
# container_name: <azure_blob_storage_container_name>
reporting:
type: file # or console, blob
base_dir: "output/${timestamp}/reports"
# connection_string: <azure_blob_storage_connection_string>
# container_name: <azure_blob_storage_container_name>
entity_extraction:
## llm: override the global llm settings for this task
## parallelization: override the global parallelization settings for this task
## async_mode: override the global async_mode settings for this task
prompt: "prompts/entity_extraction.txt"
entity_types: [organization,person,geo,event]
max_gleanings: 0
summarize_descriptions:
## llm: override the global llm settings for this task
## parallelization: override the global parallelization settings for this task
## async_mode: override the global async_mode settings for this task
prompt: "prompts/summarize_descriptions.txt"
max_length: 500
claim_extraction:
## llm: override the global llm settings for this task
## parallelization: override the global parallelization settings for this task
## async_mode: override the global async_mode settings for this task
# enabled: true
prompt: "prompts/claim_extraction.txt"
description: "Any claims or facts that could be relevant to information discovery."
max_gleanings: 0
community_report:
## llm: override the global llm settings for this task
## parallelization: override the global parallelization settings for this task
## async_mode: override the global async_mode settings for this task
prompt: "prompts/community_report.txt"
max_length: 2000
max_input_length: 8000
cluster_graph:
max_cluster_size: 10
embed_graph:
enabled: false # if true, will generate node2vec embeddings for nodes
# num_walks: 10
# walk_length: 40
# window_size: 2
# iterations: 3
# random_seed: 597832
umap:
enabled: false # if true, will generate UMAP embeddings for nodes
snapshots:
graphml: false
raw_entities: false
top_level_nodes: false
local_search:
# text_unit_prop: 0.5
# community_prop: 0.1
# conversation_history_max_turns: 5
# top_k_mapped_entities: 10
# top_k_relationships: 10
# max_tokens: 12000
global_search:
# max_tokens: 12000
# data_max_tokens: 12000
# map_max_tokens: 1000
# reduce_max_tokens: 2000
# concurrency: 32
Categories: DBA Blogs