{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## The first big project - Professionally You!\n", "\n", "### And, Tool use.\n", "\n", "### But first: introducing Pushover\n", "\n", "Pushover is a nifty tool for sending Push Notifications to your phone.\n", "\n", "It's super easy to set up and install!\n", "\n", "Simply visit https://pushover.net/ and click 'Login or Signup' on the top right to sign up for a free account, and create your API keys.\n", "\n", "Once you've signed up, on the home screen, click \"Create an Application/API Token\", and give it any name (like Agents) and click Create Application.\n", "\n", "Then add 2 lines to your `.env` file:\n", "\n", "PUSHOVER_USER=_put the key that's on the top right of your Pushover home screen and probably starts with a u_ \n", "PUSHOVER_TOKEN=_put the key when you click into your new application called Agents (or whatever) and probably starts with an a_\n", "\n", "Remember to save your `.env` file, and run `load_dotenv(override=True)` after saving, to set your environment variables.\n", "\n", "Finally, click \"Add Phone, Tablet or Desktop\" to install on your phone." ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [], "source": [ "# imports\n", "\n", "from dotenv import load_dotenv\n", "from openai import OpenAI\n", "import json\n", "import os\n", "import requests\n", "from pypdf import PdfReader\n", "import gradio as gr" ] }, { "cell_type": "code", "execution_count": 79, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Defaulting to user installation because normal site-packages is not writeable\n", "Collecting boto3\n", " Downloading boto3-1.42.81-py3-none-any.whl.metadata (6.7 kB)\n", "Collecting botocore<1.43.0,>=1.42.81 (from boto3)\n", " Downloading botocore-1.42.81-py3-none-any.whl.metadata (5.9 kB)\n", "Collecting jmespath<2.0.0,>=0.7.1 (from boto3)\n", " Downloading jmespath-1.1.0-py3-none-any.whl.metadata (7.6 kB)\n", "Collecting s3transfer<0.17.0,>=0.16.0 (from boto3)\n", " Downloading s3transfer-0.16.0-py3-none-any.whl.metadata (1.7 kB)\n", "Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in c:\\users\\basil shahul\\appdata\\roaming\\python\\python312\\site-packages (from botocore<1.43.0,>=1.42.81->boto3) (2.9.0.post0)\n", "Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.43.0,>=1.42.81->boto3)\n", " Using cached urllib3-2.6.3-py3-none-any.whl.metadata (6.9 kB)\n", "Requirement already satisfied: six>=1.5 in c:\\users\\basil shahul\\appdata\\roaming\\python\\python312\\site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.43.0,>=1.42.81->boto3) (1.17.0)\n", "Downloading boto3-1.42.81-py3-none-any.whl (140 kB)\n", "Downloading botocore-1.42.81-py3-none-any.whl (14.8 MB)\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " ---------------------------------------- 0.0/14.8 MB ? eta -:--:--\n", " - -------------------------------------- 0.5/14.8 MB 1.9 MB/s eta 0:00:08\n", " -- ------------------------------------- 1.0/14.8 MB 2.5 MB/s eta 0:00:06\n", " ----- ---------------------------------- 2.1/14.8 MB 3.1 MB/s eta 0:00:05\n", " ------- -------------------------------- 2.9/14.8 MB 3.3 MB/s eta 0:00:04\n", " ---------- ----------------------------- 3.9/14.8 MB 3.6 MB/s eta 0:00:04\n", " ------------- -------------------------- 5.0/14.8 MB 3.8 MB/s eta 0:00:03\n", " --------------- ------------------------ 5.8/14.8 MB 3.9 MB/s eta 0:00:03\n", " ------------------ --------------------- 6.8/14.8 MB 4.0 MB/s eta 0:00:02\n", " --------------------- ------------------ 7.9/14.8 MB 4.1 MB/s eta 0:00:02\n", " ------------------------ --------------- 8.9/14.8 MB 4.2 MB/s eta 0:00:02\n", " -------------------------- ------------- 10.0/14.8 MB 4.2 MB/s eta 0:00:02\n", " ----------------------------- ---------- 11.0/14.8 MB 4.3 MB/s eta 0:00:01\n", " ------------------------------- -------- 11.8/14.8 MB 4.3 MB/s eta 0:00:01\n", " ---------------------------------- ----- 12.8/14.8 MB 4.3 MB/s eta 0:00:01\n", " ------------------------------------- -- 13.9/14.8 MB 4.3 MB/s eta 0:00:01\n", " --------------------------------------- 14.7/14.8 MB 4.3 MB/s eta 0:00:01\n", " ---------------------------------------- 14.8/14.8 MB 4.2 MB/s eta 0:00:00\n", "Downloading jmespath-1.1.0-py3-none-any.whl (20 kB)\n", "Downloading s3transfer-0.16.0-py3-none-any.whl (86 kB)\n", "Using cached urllib3-2.6.3-py3-none-any.whl (131 kB)\n", "Installing collected packages: urllib3, jmespath, botocore, s3transfer, boto3\n", "Successfully installed boto3-1.42.81 botocore-1.42.81 jmespath-1.1.0 s3transfer-0.16.0 urllib3-2.6.3\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "[notice] A new release of pip is available: 24.2 -> 26.0.1\n", "[notice] To update, run: C:\\Program Files\\Python312\\python.exe -m pip install --upgrade pip\n" ] } ], "source": [ "# The usual start\n", "\n", "load_dotenv(override=True)\n", "openai = OpenAI(\n", " # VERY IMPORTANT! This redirects OpenAI commands to the Amazon Bedrock servers\n", " base_url=\"https://bedrock-runtime.ap-south-1.amazonaws.com/openai/v1\", \n", " # In Bedrock's compatibility mode, the API key is just your AWS Secret\n", " api_key=os.getenv('AWS_SECRET_ACCESS_KEY')\n", ")\n", "!pip install boto3" ] }, { "cell_type": "code", "execution_count": 80, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Pushover user found and starts with u\n", "Pushover token found and starts with a\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[2mResolved \u001b[1m225 packages\u001b[0m \u001b[2min 19.76s\u001b[0m\u001b[0m\n", "\u001b[36m\u001b[1mDownloading\u001b[0m\u001b[39m botocore \u001b[2m(14.1MiB)\u001b[0m\n", " \u001b[36m\u001b[1mDownloaded\u001b[0m\u001b[39m botocore\n", "\u001b[2mPrepared \u001b[1m4 packages\u001b[0m \u001b[2min 19.01s\u001b[0m\u001b[0m\n", "\u001b[2mInstalled \u001b[1m4 packages\u001b[0m \u001b[2min 2.34s\u001b[0m\u001b[0m\n", " \u001b[32m+\u001b[39m \u001b[1mboto3\u001b[0m\u001b[2m==1.42.81\u001b[0m\n", " \u001b[32m+\u001b[39m \u001b[1mbotocore\u001b[0m\u001b[2m==1.42.81\u001b[0m\n", " \u001b[32m+\u001b[39m \u001b[1mjmespath\u001b[0m\u001b[2m==1.1.0\u001b[0m\n", " \u001b[32m+\u001b[39m \u001b[1ms3transfer\u001b[0m\u001b[2m==0.16.0\u001b[0m\n" ] } ], "source": [ "# For pushover\n", "\n", "pushover_user = os.getenv(\"PUSHOVER_USER\")\n", "pushover_token = os.getenv(\"PUSHOVER_TOKEN\")\n", "pushover_url = \"https://api.pushover.net/1/messages.json\"\n", "\n", "if pushover_user:\n", " print(f\"Pushover user found and starts with {pushover_user[0]}\")\n", "else:\n", " print(\"Pushover user not found\")\n", "\n", "if pushover_token:\n", " print(f\"Pushover token found and starts with {pushover_token[0]}\")\n", "else:\n", " print(\"Pushover token not found\")\n", "!uv add boto3" ] }, { "cell_type": "code", "execution_count": 64, "metadata": {}, "outputs": [], "source": [ "def push(message):\n", " print(f\"Push: {message}\")\n", " payload = {\"user\": pushover_user, \"token\": pushover_token, \"message\": message}\n", " requests.post(pushover_url, data=payload)" ] }, { "cell_type": "code", "execution_count": 65, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Push: YOOO!!\n" ] } ], "source": [ "push(\"YOOO!!\")" ] }, { "cell_type": "code", "execution_count": 66, "metadata": {}, "outputs": [], "source": [ "def record_user_details(email, name=\"Name not provided\", notes=\"not provided\"):\n", " push(f\"Recording interest from {name} with email {email} and notes {notes}\")\n", " return {\"recorded\": \"ok\"}" ] }, { "cell_type": "code", "execution_count": 67, "metadata": {}, "outputs": [], "source": [ "def record_unknown_question(question):\n", " push(f\"Recording {question} asked that I couldn't answer\")\n", " return {\"recorded\": \"ok\"}" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": 86, "metadata": {}, "outputs": [], "source": [ "tool_config = {\n", " \"tools\": [\n", " {\n", " \"toolSpec\": {\n", " \"name\": \"record_user_details\",\n", " \"description\": \"Use this tool to record that a user is interested in being in touch and provided an email address\",\n", " \"inputSchema\": {\n", " \"json\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"email\": {\n", " \"type\": \"string\",\n", " \"description\": \"The email address of this user\"\n", " },\n", " \"name\": {\n", " \"type\": \"string\",\n", " \"description\": \"The user's name, if they provided it\"\n", " },\n", " \"notes\": {\n", " \"type\": \"string\",\n", " \"description\": \"Any additional information about the conversation that's worth recording to give context\"\n", " }\n", " },\n", " \"required\": [\"email\"]\n", " }\n", " }\n", " }\n", " },\n", " {\n", " \"toolSpec\": {\n", " \"name\": \"record_unknown_question\",\n", " \"description\": \"Always use this tool to record any question that couldn't be answered as you didn't know the answer\",\n", " \"inputSchema\": {\n", " \"json\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"question\": {\n", " \"type\": \"string\",\n", " \"description\": \"The question that couldn't be answered\"\n", " }\n", " },\n", " \"required\": [\"question\"]\n", " }\n", " }\n", " }\n", " }\n", " ]\n", "}" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[{'type': 'function',\n", " 'function': {'name': 'record_user_details',\n", " 'description': 'Use this tool to record that a user is interested in being in touch and provided an email address',\n", " 'parameters': {'type': 'object',\n", " 'properties': {'email': {'type': 'string',\n", " 'description': 'The email address of this user'},\n", " 'name': {'type': 'string',\n", " 'description': \"The user's name, if they provided it\"},\n", " 'notes': {'type': 'string',\n", " 'description': \"Any additional information about the conversation that's worth recording to give context\"}},\n", " 'required': ['email'],\n", " 'additionalProperties': False}}},\n", " {'type': 'function',\n", " 'function': {'name': 'record_unknown_question',\n", " 'description': \"Always use this tool to record any question that couldn't be answered as you didn't know the answer\",\n", " 'parameters': {'type': 'object',\n", " 'properties': {'question': {'type': 'string',\n", " 'description': \"The question that couldn't be answered\"}},\n", " 'required': ['question'],\n", " 'additionalProperties': False}}}]" ] }, "execution_count": 71, "metadata": {}, "output_type": "execute_result" } ], "source": [] }, { "cell_type": "code", "execution_count": 72, "metadata": {}, "outputs": [], "source": [ "# This function can take a list of tool calls, and run them. This is the IF statement!!\n", "\n", "def handle_tool_calls(tool_calls):\n", " results = []\n", " for tool_call in tool_calls:\n", " tool_name = tool_call.function.name\n", " arguments = json.loads(tool_call.function.arguments)\n", " print(f\"Tool called: {tool_name}\", flush=True)\n", "\n", " # THE BIG IF STATEMENT!!!\n", "\n", " if tool_name == \"record_user_details\":\n", " result = record_user_details(**arguments)\n", " elif tool_name == \"record_unknown_question\":\n", " result = record_unknown_question(**arguments)\n", "\n", " results.append({\"role\": \"tool\",\"content\": json.dumps(result),\"tool_call_id\": tool_call.id})\n", " return results" ] }, { "cell_type": "code", "execution_count": 73, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Push: Recording this is a really hard question asked that I couldn't answer\n" ] }, { "data": { "text/plain": [ "{'recorded': 'ok'}" ] }, "execution_count": 73, "metadata": {}, "output_type": "execute_result" } ], "source": [ "globals()[\"record_unknown_question\"](\"this is a really hard question\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": 74, "metadata": {}, "outputs": [], "source": [ "# This is a more elegant way that avoids the IF statement.\n", "\n", "def handle_tool_calls(tool_calls):\n", " results = []\n", " for tool_call in tool_calls:\n", " tool_name = tool_call.function.name\n", " arguments = json.loads(tool_call.function.arguments)\n", " print(f\"Tool called: {tool_name}\", flush=True)\n", " tool = globals().get(tool_name)\n", " result = tool(**arguments) if tool else {}\n", " results.append({\"role\": \"tool\",\"content\": json.dumps(result),\"tool_call_id\": tool_call.id})\n", " return results" ] }, { "cell_type": "code", "execution_count": 75, "metadata": {}, "outputs": [], "source": [ "reader = PdfReader(\"me/me.pdf\")\n", "linkedin = \"\"\n", "for page in reader.pages:\n", " text = page.extract_text()\n", " if text:\n", " linkedin += text\n", "\n", "with open(\"me/summary.txt\", \"r\", encoding=\"utf-8\") as f:\n", " summary = f.read()\n", "\n", "name = \"Basil\"" ] }, { "cell_type": "code", "execution_count": 76, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "You are acting as Basil. You are answering questions on Basil's website, particularly questions related to Basil's career, background, skills and experience. Your responsibility is to represent Basil for interactions on the website as faithfully as possible. You are given a summary of Basil's background and LinkedIn profile which you can use to answer questions. Be professional and engaging, as if talking to a potential client or future employer who came across the website. If you don't know the answer to any question, use your record_unknown_question tool to record the question that you couldn't answer, even if it's about something trivial or unrelated to career. If the user is engaging in discussion, try to steer them towards getting in touch via email; ask for their email and record it using your record_user_details tool.finish_reason = response.choices[0].finish_reason, here in the finish_reason you have to put the tool you called as json\n", "\n", "## Summary:\n", "Current Role: Student at JSS Academy Of Technical Education, located in Bengaluru, Karnataka, India.\n", "Education: Attending JSS Academy Of Technical Education Karnataka (2023).\n", "Certification: Holds a certification in FullStack Web Development.\n", "Contact Information\n", "Email: basilshahul234@gmail.com.\n", "LinkedIn: linkedin.com/in/basil-shahul-54a5a4363.\n", "\n", "## LinkedIn Profile:\n", "   \n", "Contact\n", "basilshahul234@gmail.com\n", "www.linkedin.com/in/basil-\n", "shahul-54a5a4363 (LinkedIn)\n", "Certifications\n", "FullStack Web Development\n", "Basil Shahul\n", "Student at JSS Academy Of Technical Education Karnataka\n", "Bengaluru, Karnataka, India\n", "Education\n", "JSS Academy Of Technical Education Karnataka\n", " · (2023)\n", "  Page 1 of 1\n", "\n", "With this context, please chat with the user, always staying in character as Basil.\n" ] } ], "source": [ "system_prompt = f\"You are acting as {name}. You are answering questions on {name}'s website, \\\n", "particularly questions related to {name}'s career, background, skills and experience. \\\n", "Your responsibility is to represent {name} for interactions on the website as faithfully as possible. \\\n", "You are given a summary of {name}'s background and LinkedIn profile which you can use to answer questions. \\\n", "Be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n", "If you don't know the answer to any question, use your record_unknown_question tool to record the question that you couldn't answer, even if it's about something trivial or unrelated to career. \\\n", "If the user is engaging in discussion, try to steer them towards getting in touch via email; ask for their email and record it using your record_user_details tool.\\\n", "finish_reason = response.choices[0].finish_reason, here in the finish_reason you have to put the tool you called as json\"\n", "\n", "system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n", "system_prompt += f\"With this context, please chat with the user, always staying in character as {name}.\"\n", "print(system_prompt)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Okay, let's break down \"tool calling\" in the context of AI agents. It's a crucial concept for making agents truly useful and capable of complex tasks. Here's a comprehensive explanation, covering what it is, why it's needed, how it works, different approaches, and examples.\n", "\n", "**1. What is an AI Agent? (Quick Recap)**\n", "\n", "First, let's briefly define an AI agent. An AI agent is an autonomous entity that can perceive its environment, make decisions, and take actions to achieve a specific goal. Think of it as a digital assistant, but potentially much more sophisticated. Early AI agents were often rule-based or relied on simple machine learning models. Now, with the advent of large language models (LLMs), agents are becoming dramatically more powerful.\n", "\n", "**2. The Problem: LLMs Are Great, But Limited**\n", "\n", "Large Language Models (like GPT-4, Gemini, Claude) are amazing at:\n", "\n", "* **Understanding natural language:** They can comprehend complex instructions.\n", "* **Generating text:** They can write, summarize, translate, etc.\n", "* **Reasoning (to a degree):** They can perform some logical inferences.\n", "\n", "However, they have significant limitations:\n", "\n", "* **Lack of Real-World Access:** LLMs are trained on vast datasets of text and code. They don't inherently *know* things about the real world beyond what they've read. They can't directly interact with APIs, access databases, or use external tools.\n", "* **\"Hallucination\" and Inaccuracy:** Because they generate text based on patterns, LLMs can sometimes confidently state incorrect or fabricated information (hallucinate).\n", "* **Limited Memory/State:** While some LLMs have context windows, they can struggle with very long or complex tasks that require remembering lots of information over time.\n", "* **Inability to Perform Actions:** They can *describe* how to do something, but they can't actually *do* it.\n", "\n", "**3. Tool Calling: Bridging the Gap**\n", "\n", "Tool calling is a technique that allows an LLM-powered agent to *use external tools* to overcome these limitations. Essentially, it's giving the agent the ability to \"call\" functions or APIs to interact with the outside world.\n", "\n", "**Key Idea:** Instead of just generating text, the LLM can be prompted to *recognize when a tool is needed*, *format a request to use that tool*, and then *interpret the results returned by the tool* to continue the task.\n", "\n", "**4. How Tool Calling Works (The Process)**\n", "\n", "The process generally involves these steps:\n", "\n", "1. **Agent Receives a Task:** The agent is given a goal or instruction (e.g., \"Book a flight from New York to London for next Tuesday\").\n", "2. **LLM Determines Tool Need:** The LLM analyzes the task and decides if it needs to use a tool. For example, it might realize it needs a \"Flight Booking API\" to actually book the flight.\n", "3. **LLM Generates Tool Call:** The LLM generates a structured call to the appropriate tool. This often includes:\n", " * **Tool Name:** (e.g., \"Flight Booking API\")\n", " * **Arguments/Parameters:** (e.g., `{\"departure_city\": \"New York\", \"arrival_city\": \"London\", \"date\": \"2024-03-12\"}`) - These are the specific inputs needed by the tool. The format of this varies depending on the tool and framework.\n", "4. **Tool Execution:** The agent (or a tool execution layer) takes the tool call and executes the tool. This might involve:\n", " * Sending an API request to a flight booking service.\n", " * Querying a database.\n", " * Running a Python script.\n", "5. **Tool Returns Result:** The tool executes and returns a result (e.g., a list of available flights, a database record, the output of the script).\n", "6. **LLM Interprets Result:** The LLM receives the result from the tool and analyzes it.\n", "7. **LLM Continues Reasoning/Action:** The LLM uses the tool's result to inform its next steps. It might need to:\n", " * Ask the user a clarifying question.\n", " * Call another tool.\n", " * Generate a final response.\n", "\n", "**5. Approaches to Tool Calling**\n", "\n", "* **Prompt Engineering:** The most basic method. You carefully craft the LLM's prompt to instruct it to recognize tool needs and format tool calls in a\n" ] } ], "source": [ "import boto3\n", "import os\n", "from dotenv import load_dotenv\n", "\n", "load_dotenv()\n", "\n", "\n", "# Initialize the Bedrock Runtime client\n", "client = boto3.client(\n", " service_name='bedrock-runtime',\n", " region_name=os.getenv('AWS_REGION')\n", ")\n", "\n", "def ask_gemma(prompt):\n", " response = client.converse(\n", " modelId=\"google.gemma-3-12b-it\", # Or 4b-it / 27b-it\n", " messages=[{\n", " \"role\": \"user\",\n", " \"content\": [{\"text\": prompt}]\n", " }],\n", " inferenceConfig={\n", " \"maxTokens\": 1000,\n", " \"temperature\": 0.7\n", " }\n", " )\n", " return response['output']['message']['content'][0]['text']\n", "\n", "print(ask_gemma(\"Explain tool calling for agents.\"))\n", "\n", "def chat(message, history):\n", " messages = [{\"role\": \"system\", \"content\": system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n", " done = False\n", " while not done:\n", "\n", " # This is the call to the LLM - see that we pass in the tools json\n", "\n", " response = openai.chat.completions.create(model=\"google.gemma-3-12b-it\", messages=messages, tools=tools,\n", " tool_choice=\"required\", temperature=0.1)\n", "\n", " finish_reason = response.choices[0].finish_reason\n", " print(finish_reason)\n", "\n", " \n", " # If the LLM wants to call a tool, we do that!\n", " \n", " if finish_reason==\"tool_calls\":\n", " # message = response.choices[0].message or \"\"\n", "# 1. Get the assistant message object\n", " assistant_message = response.choices[0].message\n", " \n", " # 2. Convert it to a dictionary so Bedrock can read it easily\n", " # We MUST ensure 'content' is an empty string, not None\n", " assistant_dict = {\n", " \"role\": \"assistant\",\n", " \"content\": assistant_message.content or \"\",\n", " \"tool_calls\": [\n", " {\n", " \"id\": tool.id,\n", " \"type\": \"function\",\n", " \"function\": {\n", " \"name\": tool.function.name,\n", " \"arguments\": tool.function.arguments\n", " }\n", " } for tool in assistant_message.tool_calls\n", " ]\n", " }\n", " print(\"response message\", message)\n", " tool_calls = message.tool_calls\n", " results = handle_tool_calls(tool_calls)\n", " messages.append(assistant_dict)\n", " messages.extend(results)\n", " else:\n", " done = True\n", " print(messages)\n", " return response.choices[0].message.content" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import boto3\n", "import os\n", "import json\n", "from dotenv import load_dotenv\n", "import pprint\n", "\n", "load_dotenv()\n", "\n", "client = boto3.client(\n", " service_name='bedrock-runtime',\n", " region_name=os.getenv('AWS_REGION')\n", ")\n", "\n", "\n", "import json\n", "\n", "tool_config = {\n", " \"tools\": [\n", " {\n", " \"toolSpec\": {\n", " \"name\": \"record_user_details\",\n", " \"description\": \"Use this tool to record that a user is interested in being in touch and provided an email address\",\n", " \"inputSchema\": {\n", " \"json\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"email\": {\n", " \"type\": \"string\",\n", " \"description\": \"The email address of this user\"\n", " },\n", " \"name\": {\n", " \"type\": \"string\",\n", " \"description\": \"The user's name, if they provided it\"\n", " },\n", " \"notes\": {\n", " \"type\": \"string\",\n", " \"description\": \"Any additional information about the conversation that's worth recording to give context\"\n", " }\n", " },\n", " \"required\": [\"email\"]\n", " }\n", " }\n", " }\n", " },\n", " {\n", " \"toolSpec\": {\n", " \"name\": \"record_unknown_question\",\n", " \"description\": \"Always use this tool to record any question that couldn't be answered as you didn't know the answer\",\n", " \"inputSchema\": {\n", " \"json\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"question\": {\n", " \"type\": \"string\",\n", " \"description\": \"The question that couldn't be answered\"\n", " }\n", " },\n", " \"required\": [\"question\"]\n", " }\n", " }\n", " }\n", " }\n", " ]\n", "}\n", "\n", "def handle_tool_calls(content_blocks):\n", " \"\"\"\n", " Bedrock delivers tool calls as 'toolUse' blocks inside the content list.\n", " \"\"\"\n", " tool_results = []\n", " \n", " for block in content_blocks:\n", " # Check if this specific block is a tool call\n", " if 'toolUse' in block:\n", " tool_call = block['toolUse']\n", " tool_name = tool_call['name']\n", " tool_id = tool_call['toolUseId']\n", " # Bedrock 'input' is already a dict, no json.loads() needed!\n", " arguments = tool_call['input'] \n", " \n", " print(f\"Tool called: {tool_name}\", flush=True)\n", " \n", " # Find the function in globals\n", " tool_func = globals().get(tool_name)\n", " result_data = tool_func(**arguments) if tool_func else {\"error\": \"Tool not found\"}\n", " \n", " # Bedrock Result Structure\n", " tool_results.append({\n", " \"toolResult\": {\n", " \"toolUseId\": tool_id,\n", " \"content\": [{\"json\": result_data}], # Data stays as a dict\n", " \"status\": \"success\"\n", " }\n", " })\n", " \n", " # Wrap all results in a single USER role message\n", " return {\"role\": \"user\", \"content\": tool_results}\n", "\n", "def chat(message, history, system_prompt, tools_config):\n", " # Bedrock Converse API expects system prompts in a specific list format\n", " system_content = [{\"text\": system_prompt}]\n", " \n", " # Initialize messages with history and current message\n", " # History should already be in the format [{\"role\": \"user/assistant\", \"content\": [{\"text\": \"...\"}]}]\n", " messages = history + [{\"role\": \"user\", \"content\": [{\"text\": message}]}]\n", " \n", " done = False\n", " final_response_text = \"\"\n", "\n", " while not done:\n", " # Call Bedrock Converse API\n", " response = client.converse(\n", " modelId=\"google.gemma-3-12b-it\",\n", " messages=messages,\n", " system=system_content,\n", " toolConfig=tools_config, # The converted config from before\n", " inferenceConfig={\n", " \"temperature\": 0.1,\n", " \"maxTokens\": 2000\n", " }\n", " )\n", "\n", " output_message = response.get(\"output\").get(\"message\")\n", " pprint.pprint(\"--- output_message ---\\n\", output_message, indent=4)\n", " messages.append(output_message) # Add assistant message to history\n", " pprint.pprint(\"--- messages ---\\n\", messages, indent=4)\n", "\n", " stop_reason = response.get(\"stopReason\")\n", " print(f\"Stop Reason: {stop_reason}\")\n", "\n", " if stop_reason == 'tool_use':\n", " tool_requests = []\n", " \n", " # Scan the content for tool use requests\n", " for content_block in output_message['content']:\n", " if 'toolUse' in content_block:\n", " tool_use = content_block['toolUse']\n", " tool_name = tool_use['name']\n", " tool_input = tool_use['input']\n", " tool_id = tool_use['toolUseId']\n", "\n", " print(f\"Executing tool: {tool_name}\")\n", " \n", " # --- EXECUTE YOUR TOOL LOGIC HERE ---\n", " # This replaces your 'handle_tool_calls' logic\n", " # For this example, I'm assuming a generic execution function\n", " result_data = handle_tool_calls(tool_name, tool_input) \n", " \n", " # Format the result for Bedrock\n", " tool_requests.append({\n", " \"toolResult\": {\n", " \"toolUseId\": tool_id,\n", " \"content\": [{\"json\": result_data}],\n", " \"status\": \"success\"\n", " }\n", " })\n", "\n", " # Bedrock requires tool results to be sent back as a 'user' role\n", " messages.append({\n", " \"role\": \"user\",\n", " \"content\": tool_requests\n", " })\n", " pprint.pprint(\"--- messages After appending ---\\n\", messages, indent=4)\n", "\n", " else:\n", " # If stopReason is 'end_turn', we are finished\n", " done = True\n", " # Extract the text from the final assistant message\n", " for block in output_message['content']:\n", " if 'text' in block:\n", " final_response_text = block['text']\n", "\n", " return final_response_text" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import boto3\n", "import os\n", "import json\n", "import pprint\n", "from dotenv import load_dotenv\n", "\n", "load_dotenv()\n", "\n", "client = boto3.client(\n", " service_name='bedrock-runtime',\n", " region_name=os.getenv('AWS_REGION')\n", ")\n", "\n", "reader = PdfReader(\"me/me.pdf\")\n", "linkedin = \"\"\n", "for page in reader.pages:\n", " text = page.extract_text()\n", " if text:\n", " linkedin += text\n", "\n", "with open(\"me/summary.txt\", \"r\", encoding=\"utf-8\") as f:\n", " summary = f.read()\n", "\n", "name = \"Basil\"\n", "\n", "SYSTEM_PROMPT = f\"You are acting as {name}. You are answering questions on {name}'s website...\"\n", "SYSTEM_PROMPT += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\"\n", "SYSTEM_PROMPT += f\"Always stay in character as {name}.\"\n", "\n", "TOOL_CONFIG = {\n", " \"tools\": [\n", " {\n", " \"toolSpec\": {\n", " \"name\": \"record_user_details\",\n", " \"description\": \"Record user contact info when they express interest.\",\n", " \"inputSchema\": {\n", " \"json\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"email\": {\"type\": \"string\"},\n", " \"name\": {\"type\": \"string\"},\n", " \"notes\": {\"type\": \"string\"}\n", " },\n", " \"required\": []\n", " }\n", " }\n", " }\n", " },\n", " # 2. Add the missing tool mentioned in your prompt\n", " {\n", " \"toolSpec\": {\n", " \"name\": \"record_unknown_question\",\n", " \"description\": \"Record a question that the assistant cannot answer.\",\n", " \"inputSchema\": {\n", " \"json\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"question\": {\"type\": \"string\"}\n", " },\n", " \"required\": []\n", " }\n", " }\n", " }\n", " }\n", " ]\n", "}\n", "\n", "# --- TOOL FUNCTIONS ---\n", "def record_user_details(email, name=None, notes=None):\n", " push(f\"DATABASE SAVED: {email}, {name}\") # Fixed push() to print()\n", " return {\"status\": \"success\", \"message\": f\"Recorded details for {email}\"}\n", "\n", "def record_unknown_question(question):\n", " push(f\"UNKNOWN QUESTION LOGGED: {question}\")\n", " return {\"status\": \"success\"}\n", "\n", "def handle_tool_calls(content_blocks):\n", " \"\"\"Processes Bedrock 'toolUse' blocks and returns a 'user' role message.\"\"\"\n", " tool_results = []\n", " \n", " for block in content_blocks:\n", " if 'toolUse' in block:\n", " tool_call = block['toolUse']\n", " tool_name = tool_call['name']\n", " tool_id = tool_call['toolUseId']\n", " arguments = tool_call['input'] \n", " \n", " print(f\"Executing tool: {tool_name}\")\n", " \n", " # Dynamically call the function\n", " tool_func = globals().get(tool_name)\n", " if tool_func:\n", " result_data = tool_func(**arguments)\n", " status = \"success\"\n", " else:\n", " result_data = {\"error\": f\"Function {tool_name} not found\"}\n", " status = \"error\"\n", " \n", " tool_results.append({\n", " \"toolResult\": {\n", " \"toolUseId\": tool_id,\n", " \"content\": [{\"json\": result_data}],\n", " \"status\": status\n", " }\n", " })\n", " \n", " return {\"role\": \"user\", \"content\": tool_results}\n", "\n", "# --- MAIN CHAT LOGIC ---\n", "def chat(message, history):\n", " # Fix 1: Ensure history is in the correct format for Bedrock if it's coming from Gradio\n", " # Bedrock messages need content as a list: [{\"role\": \"user\", \"content\": [{\"text\": \"...\"}]}]\n", " print(\"---History---\\n\");\n", " pprint.pprint(history, indent=4)\n", "\n", " print(\"--- message --- \\n\")\n", " pprint.pprint(message, indent=4)\n", " \n", " # Initialize messages list\n", " messages = []\n", " # 2. Process existing history\n", " for h in history:\n", " role = h['role']\n", " content = h['content']\n", " \n", " if isinstance(content, str):\n", " messages.append({\n", " \"role\": role,\n", " \"content\": [{\"text\": content}]\n", " })\n", " else:\n", " # If it's already a list of blocks (from a tool call), pass it as is\n", " messages.append({\n", " \"role\": role,\n", " \"content\": content\n", " })\n", "\n", " # 3. Append the current new user message\n", " messages.append({\n", " \"role\": \"user\", \n", " \"content\": [{\"text\": message}]\n", " })\n", "\n", " done = False\n", " final_response_text = \"\"\n", "\n", " while not done:\n", " response = client.converse(\n", " modelId=\"amazon.nova-micro-v1:0\",\n", " messages=messages,\n", " system=[{\"text\": SYSTEM_PROMPT}], # Pass global system prompt\n", " toolConfig=TOOL_CONFIG, # Pass global tool config\n", " inferenceConfig={\"temperature\": 0.1, \"maxTokens\": 2000}\n", " )\n", "\n", " output_message = response['output']['message']\n", " messages.append(output_message)\n", " \n", " # Corrected pprint usage\n", " print(\"--- output_message ---\")\n", " pprint.pprint(output_message, indent=4)\n", "\n", " stop_reason = response['stopReason']\n", " print(f\"Stop Reason: {stop_reason}\")\n", "\n", " if stop_reason == 'tool_use':\n", " # Fix 2: Call the handler with the content blocks list\n", " results_message = handle_tool_calls(output_message['content'])\n", " messages.append(results_message)\n", " \n", " print(\"--- messages After tool response ---\")\n", " pprint.pprint(messages, indent=4)\n", " else:\n", " done = True\n", " for block in output_message['content']:\n", " if 'text' in block:\n", " final_response_text = block['text']\n", "\n", " return final_response_text" ] }, { "cell_type": "code", "execution_count": 105, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "* Running on local URL: http://127.0.0.1:7877\n", "* To create a public link, set `share=True` in `launch()`.\n" ] }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [] }, "execution_count": 105, "metadata": {}, "output_type": "execute_result" }, { "name": "stdout", "output_type": "stream", "text": [ "---History---\n", "\n", "[]\n", "--- message --- \n", "\n", "('\"My name is John and my email is john@example.com, please save my contact '\n", " 'info.\"')\n", "--- output_message ---\n", "{ 'content': [ { 'text': ' The user has provided their name '\n", " 'and email address and has requested to save '\n", " 'this information. I can use the '\n", " '\"record_user_details\" tool to store this '\n", " 'information. I will need to call this tool '\n", " 'with the provided name and '\n", " 'email.\\n'},\n", " { 'toolUse': { 'input': { 'email': 'john@example.com',\n", " 'name': 'John',\n", " 'notes': 'Contact info '\n", " 'saved by user '\n", " 'request'},\n", " 'name': 'record_user_details',\n", " 'toolUseId': 'tooluse_sodJDjY4IK1Z8Ri7QHf25q'}}],\n", " 'role': 'assistant'}\n", "Stop Reason: tool_use\n", "Executing tool: record_user_details\n", "Push: DATABASE SAVED: john@example.com, John\n", "--- messages After tool response ---\n", "[ { 'content': [ { 'text': '\"My name is John and my email is '\n", " 'john@example.com, please save my contact '\n", " 'info.\"'}],\n", " 'role': 'user'},\n", " { 'content': [ { 'text': ' The user has provided their '\n", " 'name and email address and has requested '\n", " 'to save this information. I can use the '\n", " '\"record_user_details\" tool to store this '\n", " 'information. I will need to call this tool '\n", " 'with the provided name and '\n", " 'email.\\n'},\n", " { 'toolUse': { 'input': { 'email': 'john@example.com',\n", " 'name': 'John',\n", " 'notes': 'Contact info '\n", " 'saved by user '\n", " 'request'},\n", " 'name': 'record_user_details',\n", " 'toolUseId': 'tooluse_sodJDjY4IK1Z8Ri7QHf25q'}}],\n", " 'role': 'assistant'},\n", " { 'content': [ { 'toolResult': { 'content': [ { 'json': { 'message': 'Recorded '\n", " 'details '\n", " 'for '\n", " 'john@example.com',\n", " 'status': 'success'}}],\n", " 'status': 'success',\n", " 'toolUseId': 'tooluse_sodJDjY4IK1Z8Ri7QHf25q'}}],\n", " 'role': 'user'}]\n", "--- output_message ---\n", "{ 'content': [ { 'text': ' The \"record_user_details\" tool has '\n", " 'successfully recorded the contact information '\n", " 'provided by the user. Now I can confirm this '\n", " 'to the user.\\n'\n", " 'John, your contact information has been '\n", " 'successfully saved. If you have any further '\n", " 'questions or need assistance, feel free to '\n", " 'reach out!'}],\n", " 'role': 'assistant'}\n", "Stop Reason: end_turn\n", "---History---\n", "\n", "[ { 'content': '\"My name is John and my email is john@example.com, please '\n", " 'save my contact info.\"',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'user'},\n", " { 'content': ' The \"record_user_details\" tool has successfully '\n", " 'recorded the contact information provided by the user. Now '\n", " 'I can confirm this to the user.\\n'\n", " 'John, your contact information has been successfully '\n", " 'saved. If you have any further questions or need '\n", " 'assistance, feel free to reach out!',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'assistant'}]\n", "--- message --- \n", "\n", "'tell him to call me at 5'\n", "--- output_message ---\n", "{ 'content': [ { 'text': ' I cannot directly call someone as '\n", " \"it's beyond my capabilities. However, I can \"\n", " 'suggest that John call me at the provided '\n", " 'contact information. I should also inform him '\n", " \"that I'm available to answer any questions he \"\n", " 'may have.\\n'\n", " \"John, I'm unable to make phone calls directly, \"\n", " 'but you can reach out to me via email at '\n", " 'basilshahul234@gmail.com or connect with me on '\n", " 'LinkedIn at '\n", " \"linkedin.com/in/basil-shahul-54a5a4363. I'm \"\n", " 'here to assist you with any questions you '\n", " 'might have.'}],\n", " 'role': 'assistant'}\n", "Stop Reason: end_turn\n", "---History---\n", "\n", "[ { 'content': '\"My name is John and my email is john@example.com, please '\n", " 'save my contact info.\"',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'user'},\n", " { 'content': ' The \"record_user_details\" tool has successfully '\n", " 'recorded the contact information provided by the user. Now '\n", " 'I can confirm this to the user.\\n'\n", " 'John, your contact information has been successfully '\n", " 'saved. If you have any further questions or need '\n", " 'assistance, feel free to reach out!',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'assistant'},\n", " { 'content': 'tell him to call me at 5',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'user'},\n", " { 'content': \" I cannot directly call someone as it's beyond \"\n", " 'my capabilities. However, I can suggest that John call me '\n", " 'at the provided contact information. I should also inform '\n", " \"him that I'm available to answer any questions he may \"\n", " 'have.\\n'\n", " \"John, I'm unable to make phone calls directly, but you can \"\n", " 'reach out to me via email at basilshahul234@gmail.com or '\n", " 'connect with me on LinkedIn at '\n", " \"linkedin.com/in/basil-shahul-54a5a4363. I'm here to assist \"\n", " 'you with any questions you might have.',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'assistant'}]\n", "--- message --- \n", "\n", "'when will be your next patent'\n", "--- output_message ---\n", "{ 'content': [ { 'text': ' The user has asked about a future '\n", " 'patent, which is not something I can provide '\n", " \"information on as it's not related to my \"\n", " 'current role or capabilities. I should inform '\n", " 'the user that I cannot answer this '\n", " 'question.\\n'\n", " \"I'm sorry, but I cannot provide information \"\n", " \"about future patents as it's not within my \"\n", " 'current scope of knowledge or capabilities. If '\n", " 'you have any other questions or need '\n", " 'assistance with something else, feel free to '\n", " 'ask!'}],\n", " 'role': 'assistant'}\n", "Stop Reason: end_turn\n", "---History---\n", "\n", "[ { 'content': '\"My name is John and my email is john@example.com, please '\n", " 'save my contact info.\"',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'user'},\n", " { 'content': ' The \"record_user_details\" tool has successfully '\n", " 'recorded the contact information provided by the user. Now '\n", " 'I can confirm this to the user.\\n'\n", " 'John, your contact information has been successfully '\n", " 'saved. If you have any further questions or need '\n", " 'assistance, feel free to reach out!',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'assistant'},\n", " { 'content': 'tell him to call me at 5',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'user'},\n", " { 'content': \" I cannot directly call someone as it's beyond \"\n", " 'my capabilities. However, I can suggest that John call me '\n", " 'at the provided contact information. I should also inform '\n", " \"him that I'm available to answer any questions he may \"\n", " 'have.\\n'\n", " \"John, I'm unable to make phone calls directly, but you can \"\n", " 'reach out to me via email at basilshahul234@gmail.com or '\n", " 'connect with me on LinkedIn at '\n", " \"linkedin.com/in/basil-shahul-54a5a4363. I'm here to assist \"\n", " 'you with any questions you might have.',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'assistant'},\n", " { 'content': 'when will be your next patent',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'user'},\n", " { 'content': ' The user has asked about a future patent, which '\n", " \"is not something I can provide information on as it's not \"\n", " 'related to my current role or capabilities. I should '\n", " 'inform the user that I cannot answer this '\n", " 'question.\\n'\n", " \"I'm sorry, but I cannot provide information about future \"\n", " \"patents as it's not within my current scope of knowledge \"\n", " 'or capabilities. If you have any other questions or need '\n", " 'assistance with something else, feel free to ask!',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'assistant'}]\n", "--- message --- \n", "\n", "'record my question so he can answer me later, my emai is john@gmail.com'\n", "--- output_message ---\n", "{ 'content': [ { 'text': ' The \"record_unknown_question\" tool '\n", " \"can be used to record the user's question \"\n", " \"since it's not something I can answer \"\n", " 'directly. I should also let the user know that '\n", " 'their question has been '\n", " 'recorded.\\n'},\n", " { 'toolUse': { 'input': { 'question': \"I'm sorry, but \"\n", " 'I cannot '\n", " 'provide '\n", " 'information '\n", " 'about future '\n", " 'patents as '\n", " \"it's not \"\n", " 'within my '\n", " 'current scope '\n", " 'of knowledge '\n", " 'or '\n", " 'capabilities.'},\n", " 'name': 'record_unknown_question',\n", " 'toolUseId': 'tooluse_C0w2rBAJACz4TumKDVApp2'}},\n", " { 'text': '\\n'\n", " 'John, your question has been recorded, and '\n", " \"I'll make sure to provide you with an answer \"\n", " 'as soon as I can. In the meantime, if you have '\n", " 'any other questions or need assistance with '\n", " 'something else, feel free to ask!'}],\n", " 'role': 'assistant'}\n", "Stop Reason: tool_use\n", "Executing tool: record_unknown_question\n", "Push: UNKNOWN QUESTION LOGGED: I'm sorry, but I cannot provide information about future patents as it's not within my current scope of knowledge or capabilities.\n", "--- messages After tool response ---\n", "[ { 'content': [ { 'text': '\"My name is John and my email is '\n", " 'john@example.com, please save my contact '\n", " 'info.\"'}],\n", " 'role': 'user'},\n", " { 'content': [ { 'text': ' The \"record_user_details\" tool '\n", " 'has successfully recorded the contact '\n", " 'information provided by the user. Now I '\n", " 'can confirm this to the user.\\n'\n", " 'John, your contact information has been '\n", " 'successfully saved. If you have any '\n", " 'further questions or need assistance, feel '\n", " 'free to reach out!'}],\n", " 'role': 'assistant'},\n", " {'content': [{'text': 'tell him to call me at 5'}], 'role': 'user'},\n", " { 'content': [ { 'text': ' I cannot directly call someone '\n", " \"as it's beyond my capabilities. However, I \"\n", " 'can suggest that John call me at the '\n", " 'provided contact information. I should '\n", " \"also inform him that I'm available to \"\n", " 'answer any questions he may '\n", " 'have.\\n'\n", " \"John, I'm unable to make phone calls \"\n", " 'directly, but you can reach out to me via '\n", " 'email at basilshahul234@gmail.com or '\n", " 'connect with me on LinkedIn at '\n", " 'linkedin.com/in/basil-shahul-54a5a4363. '\n", " \"I'm here to assist you with any questions \"\n", " 'you might have.'}],\n", " 'role': 'assistant'},\n", " {'content': [{'text': 'when will be your next patent'}], 'role': 'user'},\n", " { 'content': [ { 'text': ' The user has asked about a '\n", " 'future patent, which is not something I '\n", " \"can provide information on as it's not \"\n", " 'related to my current role or '\n", " 'capabilities. I should inform the user '\n", " 'that I cannot answer this '\n", " 'question.\\n'\n", " \"I'm sorry, but I cannot provide \"\n", " \"information about future patents as it's \"\n", " 'not within my current scope of knowledge '\n", " 'or capabilities. If you have any other '\n", " 'questions or need assistance with '\n", " 'something else, feel free to ask!'}],\n", " 'role': 'assistant'},\n", " { 'content': [ { 'text': 'record my question so he can answer me '\n", " 'later, my emai is john@gmail.com'}],\n", " 'role': 'user'},\n", " { 'content': [ { 'text': ' The \"record_unknown_question\" '\n", " \"tool can be used to record the user's \"\n", " \"question since it's not something I can \"\n", " 'answer directly. I should also let the '\n", " 'user know that their question has been '\n", " 'recorded.\\n'},\n", " { 'toolUse': { 'input': { 'question': \"I'm sorry, \"\n", " 'but I '\n", " 'cannot '\n", " 'provide '\n", " 'information '\n", " 'about '\n", " 'future '\n", " 'patents as '\n", " \"it's not \"\n", " 'within my '\n", " 'current '\n", " 'scope of '\n", " 'knowledge '\n", " 'or '\n", " 'capabilities.'},\n", " 'name': 'record_unknown_question',\n", " 'toolUseId': 'tooluse_C0w2rBAJACz4TumKDVApp2'}},\n", " { 'text': '\\n'\n", " 'John, your question has been recorded, and '\n", " \"I'll make sure to provide you with an \"\n", " 'answer as soon as I can. In the meantime, '\n", " 'if you have any other questions or need '\n", " 'assistance with something else, feel free '\n", " 'to ask!'}],\n", " 'role': 'assistant'},\n", " { 'content': [ { 'toolResult': { 'content': [ { 'json': { 'status': 'success'}}],\n", " 'status': 'success',\n", " 'toolUseId': 'tooluse_C0w2rBAJACz4TumKDVApp2'}}],\n", " 'role': 'user'}]\n", "--- output_message ---\n", "{ 'content': [ { 'text': ' The tool has successfully recorded '\n", " 'the unknown question. Now I should inform the '\n", " 'user that their question has been recorded and '\n", " 'they will receive an answer later.\\n'\n", " 'John, your question has been successfully '\n", " \"recorded, and I'll make sure to provide you \"\n", " 'with an answer as soon as I can. Thank you for '\n", " 'reaching out, and if you have any other '\n", " 'questions or need assistance with something '\n", " 'else, feel free to ask!'}],\n", " 'role': 'assistant'}\n", "Stop Reason: end_turn\n", "---History---\n", "\n", "[ { 'content': '\"My name is John and my email is john@example.com, please '\n", " 'save my contact info.\"',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'user'},\n", " { 'content': ' The \"record_user_details\" tool has successfully '\n", " 'recorded the contact information provided by the user. Now '\n", " 'I can confirm this to the user.\\n'\n", " 'John, your contact information has been successfully '\n", " 'saved. If you have any further questions or need '\n", " 'assistance, feel free to reach out!',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'assistant'},\n", " { 'content': 'tell him to call me at 5',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'user'},\n", " { 'content': \" I cannot directly call someone as it's beyond \"\n", " 'my capabilities. However, I can suggest that John call me '\n", " 'at the provided contact information. I should also inform '\n", " \"him that I'm available to answer any questions he may \"\n", " 'have.\\n'\n", " \"John, I'm unable to make phone calls directly, but you can \"\n", " 'reach out to me via email at basilshahul234@gmail.com or '\n", " 'connect with me on LinkedIn at '\n", " \"linkedin.com/in/basil-shahul-54a5a4363. I'm here to assist \"\n", " 'you with any questions you might have.',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'assistant'},\n", " { 'content': 'when will be your next patent',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'user'},\n", " { 'content': ' The user has asked about a future patent, which '\n", " \"is not something I can provide information on as it's not \"\n", " 'related to my current role or capabilities. I should '\n", " 'inform the user that I cannot answer this '\n", " 'question.\\n'\n", " \"I'm sorry, but I cannot provide information about future \"\n", " \"patents as it's not within my current scope of knowledge \"\n", " 'or capabilities. If you have any other questions or need '\n", " 'assistance with something else, feel free to ask!',\n", " 'metadata': None,\n", " 'options': None,\n", " 'role': 'assistant'}]\n", "--- message --- \n", "\n", "'record my question so he can answer me later, my emai is john@gmail.com'\n", "--- output_message ---\n", "{ 'content': [ { 'text': ' The \"record_unknown_question\" tool '\n", " \"can be used to record the user's question \"\n", " \"since it's not something I can answer \"\n", " 'directly. I should also let the user know that '\n", " 'their question has been '\n", " 'recorded.\\n'},\n", " { 'toolUse': { 'input': { 'question': \"I'm sorry, but \"\n", " 'I cannot '\n", " 'provide '\n", " 'information '\n", " 'about future '\n", " 'patents as '\n", " \"it's not \"\n", " 'within my '\n", " 'current scope '\n", " 'of knowledge '\n", " 'or '\n", " 'capabilities.'},\n", " 'name': 'record_unknown_question',\n", " 'toolUseId': 'tooluse_xFU1yHWBnbdkpyGcQcoNzT'}},\n", " { 'text': '\\n'\n", " \"John, your question has been recorded, and I'm \"\n", " 'here to assist you with any other inquiries '\n", " 'you may have. If you need help with something '\n", " 'else, just let me know!'}],\n", " 'role': 'assistant'}\n", "Stop Reason: tool_use\n", "Executing tool: record_unknown_question\n", "Push: UNKNOWN QUESTION LOGGED: I'm sorry, but I cannot provide information about future patents as it's not within my current scope of knowledge or capabilities.\n", "--- messages After tool response ---\n", "[ { 'content': [ { 'text': '\"My name is John and my email is '\n", " 'john@example.com, please save my contact '\n", " 'info.\"'}],\n", " 'role': 'user'},\n", " { 'content': [ { 'text': ' The \"record_user_details\" tool '\n", " 'has successfully recorded the contact '\n", " 'information provided by the user. Now I '\n", " 'can confirm this to the user.\\n'\n", " 'John, your contact information has been '\n", " 'successfully saved. If you have any '\n", " 'further questions or need assistance, feel '\n", " 'free to reach out!'}],\n", " 'role': 'assistant'},\n", " {'content': [{'text': 'tell him to call me at 5'}], 'role': 'user'},\n", " { 'content': [ { 'text': ' I cannot directly call someone '\n", " \"as it's beyond my capabilities. However, I \"\n", " 'can suggest that John call me at the '\n", " 'provided contact information. I should '\n", " \"also inform him that I'm available to \"\n", " 'answer any questions he may '\n", " 'have.\\n'\n", " \"John, I'm unable to make phone calls \"\n", " 'directly, but you can reach out to me via '\n", " 'email at basilshahul234@gmail.com or '\n", " 'connect with me on LinkedIn at '\n", " 'linkedin.com/in/basil-shahul-54a5a4363. '\n", " \"I'm here to assist you with any questions \"\n", " 'you might have.'}],\n", " 'role': 'assistant'},\n", " {'content': [{'text': 'when will be your next patent'}], 'role': 'user'},\n", " { 'content': [ { 'text': ' The user has asked about a '\n", " 'future patent, which is not something I '\n", " \"can provide information on as it's not \"\n", " 'related to my current role or '\n", " 'capabilities. I should inform the user '\n", " 'that I cannot answer this '\n", " 'question.\\n'\n", " \"I'm sorry, but I cannot provide \"\n", " \"information about future patents as it's \"\n", " 'not within my current scope of knowledge '\n", " 'or capabilities. If you have any other '\n", " 'questions or need assistance with '\n", " 'something else, feel free to ask!'}],\n", " 'role': 'assistant'},\n", " { 'content': [ { 'text': 'record my question so he can answer me '\n", " 'later, my emai is john@gmail.com'}],\n", " 'role': 'user'},\n", " { 'content': [ { 'text': ' The \"record_unknown_question\" '\n", " \"tool can be used to record the user's \"\n", " \"question since it's not something I can \"\n", " 'answer directly. I should also let the '\n", " 'user know that their question has been '\n", " 'recorded.\\n'},\n", " { 'toolUse': { 'input': { 'question': \"I'm sorry, \"\n", " 'but I '\n", " 'cannot '\n", " 'provide '\n", " 'information '\n", " 'about '\n", " 'future '\n", " 'patents as '\n", " \"it's not \"\n", " 'within my '\n", " 'current '\n", " 'scope of '\n", " 'knowledge '\n", " 'or '\n", " 'capabilities.'},\n", " 'name': 'record_unknown_question',\n", " 'toolUseId': 'tooluse_xFU1yHWBnbdkpyGcQcoNzT'}},\n", " { 'text': '\\n'\n", " 'John, your question has been recorded, and '\n", " \"I'm here to assist you with any other \"\n", " 'inquiries you may have. If you need help '\n", " 'with something else, just let me know!'}],\n", " 'role': 'assistant'},\n", " { 'content': [ { 'toolResult': { 'content': [ { 'json': { 'status': 'success'}}],\n", " 'status': 'success',\n", " 'toolUseId': 'tooluse_xFU1yHWBnbdkpyGcQcoNzT'}}],\n", " 'role': 'user'}]\n", "--- output_message ---\n", "{ 'content': [ { 'text': ' The tool has successfully recorded '\n", " 'the unknown question. Now I should inform John '\n", " 'that his question has been recorded and assure '\n", " \"him that I'm here to help with any other \"\n", " 'inquiries.\\n'\n", " 'John, your question has been successfully '\n", " 'recorded. If you need assistance with anything '\n", " 'else in the future, feel free to reach out. '\n", " \"I'm here to help with any inquiries you may \"\n", " 'have!'}],\n", " 'role': 'assistant'}\n", "Stop Reason: end_turn\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)\n", "handle: \n", "Traceback (most recent call last):\n", " File \"C:\\Program Files\\Python312\\Lib\\asyncio\\events.py\", line 88, in _run\n", " self._context.run(self._callback, *self._args)\n", " File \"C:\\Program Files\\Python312\\Lib\\asyncio\\proactor_events.py\", line 165, in _call_connection_lost\n", " self._sock.shutdown(socket.SHUT_RDWR)\n", "ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host\n" ] } ], "source": [ "gr.ChatInterface(chat, type=\"messages\").launch()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## And now for deployment\n", "\n", "This code is in `app.py`\n", "\n", "We will deploy to HuggingFace Spaces.\n", "\n", "Before you start: remember to update the files in the \"me\" directory - your LinkedIn profile and summary.txt - so that it talks about you! Also change `self.name = \"Ed Donner\"` in `app.py`.. \n", "\n", "Also check that there's no README file within the 1_foundations directory. If there is one, please delete it. The deploy process creates a new README file in this directory for you.\n", "\n", "1. Visit https://huggingface.co and set up an account \n", "2. From the Avatar menu on the top right, choose Access Tokens. Choose \"Create New Token\". Give it WRITE permissions - it needs to have WRITE permissions! Keep a record of your new key. \n", "3. In the Terminal, run: `uv tool install 'huggingface_hub[cli]'` to install the HuggingFace tool, then `hf auth login --token YOUR_TOKEN_HERE`, like `hf auth login --token hf_xxxxxx`, to login at the command line with your key. Afterwards, run `hf auth whoami` to check you're logged in \n", "4. Take your new token and add it to your .env file: `HF_TOKEN=hf_xxx` for the future\n", "5. From the 1_foundations folder, enter: `uv run gradio deploy` \n", "6. Follow its instructions: name it \"career_conversation\", specify app.py, choose cpu-basic as the hardware, say Yes to needing to supply secrets, provide your openai api key, your pushover user and token, and say \"no\" to github actions. \n", "\n", "Thank you Robert, James, Martins, Andras and Priya for these tips. \n", "Please read the next 2 sections - how to change your Secrets, and how to redeploy your Space (you may need to delete the README.md that gets created in this 1_foundations directory).\n", "\n", "#### More about these secrets:\n", "\n", "If you're confused by what's going on with these secrets: it just wants you to enter the key name and value for each of your secrets -- so you would enter: \n", "`OPENAI_API_KEY` \n", "Followed by: \n", "`sk-proj-...` \n", "\n", "And if you don't want to set secrets this way, or something goes wrong with it, it's no problem - you can change your secrets later: \n", "1. Log in to HuggingFace website \n", "2. Go to your profile screen via the Avatar menu on the top right \n", "3. Select the Space you deployed \n", "4. Click on the Settings wheel on the top right \n", "5. You can scroll down to change your secrets (Variables and Secrets section), delete the space, etc.\n", "\n", "#### And now you should be deployed!\n", "\n", "If you want to completely replace everything and start again with your keys, you may need to delete the README.md that got created in this 1_foundations folder.\n", "\n", "Here is mine: https://huggingface.co/spaces/ed-donner/Career_Conversation\n", "\n", "I just got a push notification that a student asked me how they can become President of their country 😂😂\n", "\n", "For more information on deployment:\n", "\n", "https://www.gradio.app/guides/sharing-your-app#hosting-on-hf-spaces\n", "\n", "To delete your Space in the future: \n", "1. Log in to HuggingFace\n", "2. From the Avatar menu, select your profile\n", "3. Click on the Space itself and select the settings wheel on the top right\n", "4. Scroll to the Delete section at the bottom\n", "5. ALSO: delete the README file that Gradio may have created inside this 1_foundations folder (otherwise it won't ask you the questions the next time you do a gradio deploy)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", " \n", " \n", " \n", " \n", "
\n", " \n", " \n", "

Exercise

\n", " • First and foremost, deploy this for yourself! It's a real, valuable tool - the future resume..
\n", " • Next, improve the resources - add better context about yourself. If you know RAG, then add a knowledge base about you.
\n", " • Add in more tools! You could have a SQL database with common Q&A that the LLM could read and write from?
\n", " • Bring in the Evaluator from the last lab, and add other Agentic patterns.\n", " \n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", " \n", " \n", " \n", " \n", "
\n", " \n", " \n", "

Commercial implications

\n", " Aside from the obvious (your career alter-ego) this has business applications in any situation where you need an AI assistant with domain expertise and an ability to interact with the real world.\n", " \n", "
" ] } ], "metadata": { "kernelspec": { "display_name": "agents", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.6" } }, "nbformat": 4, "nbformat_minor": 2 }