Skip to content

2
22_23-J 21
Commit 5d53e813 authored by Karunarathna K.M.D.Y.K's avatar Karunarathna K.M.D.Y.K
Browse files
Options

Merge branch 'IT19975382' into 'master' 

It19975382

See merge request !4
parents 069ed7eb fa272058
Hide whitespace changes
Inline Side-by-side
Showing with 4394 additions and 67 deletions
BackEnd/ChatBot.ipynb 0 → 100644
View file @ 5d53e813
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "Zf_tbdNcTQ1-"
},
"source": [
"## 💬 Connecting to G_Drive and Obtain Data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "PVmB1RkwwJ0c",
"outputId": "030cbf01-cb90-4f84-cb6a-077811a80b89"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
"Collecting tflearn\n",
" Downloading tflearn-0.5.0.tar.gz (107 kB)\n",
"\u001b[K |████████████████████████████████| 107 kB 16.1 MB/s \n",
"\u001b[?25hRequirement already satisfied: numpy in /usr/local/lib/python3.8/dist-packages (from tflearn) (1.21.6)\n",
"Requirement already satisfied: six in /usr/local/lib/python3.8/dist-packages (from tflearn) (1.15.0)\n",
"Requirement already satisfied: Pillow in /usr/local/lib/python3.8/dist-packages (from tflearn) (7.1.2)\n",
"Building wheels for collected packages: tflearn\n",
" Building wheel for tflearn (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for tflearn: filename=tflearn-0.5.0-py3-none-any.whl size=127299 sha256=7aa8c9d51bda87a05a9bfe4fb536fc09da67d3f073bfa7d5415d3b5ba9013199\n",
" Stored in directory: /root/.cache/pip/wheels/65/9b/15/cb1e6b279c14ed897530d15cfd7da8e3df8a947e593f5cfe59\n",
"Successfully built tflearn\n",
"Installing collected packages: tflearn\n",
"Successfully installed tflearn-0.5.0\n"
]
}
],
"source": [
"!pip install tflearn"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "9ISlJ0VvEnTR",
"outputId": "ef142319-fc2f-4310-d908-ebb5d7987434"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Mounted at /content/drive\n"
]
}
],
"source": [
"from google.colab import drive\n",
"drive.mount('/content/drive')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "P16pAOJFFAuY"
},
"outputs": [],
"source": [
"train_data_file = '/content/drive/MyDrive/RP_GoviMitura/Kavee - ChatBot/DataSets/intents.json'"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## 💬 Data_Preprocessing and Model Training"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#Used in Tensorflow Model\n",
"import numpy as np\n",
"import tensorflow as tf\n",
"import tflearn\n",
"import random\n",
"\n",
"#Usde to for Contextualisation and Other NLP Tasks.\n",
"import nltk\n",
"from nltk.stem.lancaster import LancasterStemmer\n",
"stemmer = LancasterStemmer()\n",
"\n",
"#Other\n",
"import json\n",
"import pickle\n",
"import warnings\n",
"warnings.filterwarnings(\"ignore\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(\"Processing the Intents.....\")\n",
"with open(train_data_file) as json_data:\n",
" intents = json.load(json_data)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"nltk.download('punkt')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"words = []\n",
"classes = []\n",
"documents = []\n",
"ignore_words = ['?']\n",
"print(\"Looping through the Intents to Convert them to words, classes, documents and ignore_words.......\")\n",
"for intent in intents['intents']:\n",
" for pattern in intent['patterns']:\n",
" # tokenize each word in the sentence\n",
" w = nltk.word_tokenize(pattern)\n",
" # add to our words list\n",
" words.extend(w)\n",
" # add to documents in our corpus\n",
" documents.append((w, intent['tag']))\n",
" # add to our classes list\n",
" if intent['tag'] not in classes:\n",
" classes.append(intent['tag'])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(\"Stemming, Lowering and Removing Duplicates.......\")\n",
"words = [stemmer.stem(w.lower()) for w in words if w not in ignore_words]\n",
"words = sorted(list(set(words)))\n",
"\n",
"# remove duplicates\n",
"classes = sorted(list(set(classes)))\n",
"\n",
"print (len(documents), \"documents\")\n",
"print (len(classes), \"classes\", classes)\n",
"print (len(words), \"unique stemmed words\", words)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(documents)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(\"Creating the Data for our Model.....\")\n",
"training = []\n",
"output = []\n",
"print(\"Creating an List (Empty) for Output.....\")\n",
"output_empty = [0] * len(classes)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(\"Creating Traning Set, Bag of Words for our Model....\")\n",
"for doc in documents:\n",
" # initialize our bag of words\n",
" bag = []\n",
" # list of tokenized words for the pattern\n",
" pattern_words = doc[0]\n",
" #print('pattern words - ', pattern_words)\n",
" # stem each word\n",
" pattern_words = [stemmer.stem(word.lower()) for word in pattern_words]\n",
" #print('pattern words - ', pattern_words)\n",
" \n",
" for w in words: # create our bag of words array\n",
" bag.append(1) if w in pattern_words else bag.append(0)\n",
"\n",
" # output is a '0' for each tag and '1' for current tag\n",
" output_row = list(output_empty)\n",
" output_row[classes.index(doc[1])] = 1\n",
" #print('output row - ', output_row)\n",
" #print('BAG - ', bag)\n",
"\n",
" training.append([bag, output_row])\n",
" #print(training)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from tensorflow.python.framework import ops"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(\"Shuffling Randomly and Converting into Numpy Array for Faster Processing......\")\n",
"random.shuffle(training)\n",
"training = np.array(training)\n",
"\n",
"print(\"Creating Train and Test Lists.....\")\n",
"train_x = list(training[:,0])\n",
"train_y = list(training[:,1])\n",
"print(\"Building Neural Network for Out Chatbot to be Contextual....\")\n",
"print(\"Resetting graph data....\")\n",
"ops.reset_default_graph()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"net = tflearn.input_data(shape=[None, len(train_x[0])])\n",
"net = tflearn.fully_connected(net, 8)\n",
"net = tflearn.fully_connected(net, 16)\n",
"net = tflearn.fully_connected(net, 8)\n",
"net = tflearn.fully_connected(net, len(train_y[0]), activation='softmax')\n",
"net = tflearn.regression(net)\n",
"print(\"Training....\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"model = tflearn.DNN(net, tensorboard_dir='tflearn_logs')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(\"Training the Model.......\")\n",
"model.fit(train_x, train_y, n_epoch=1000, batch_size=32, show_metric=True)\n",
"print(\"Saving the Model.......\")\n",
"model.save('model.tflearn')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(\"Pickle is also Saved..........\")\n",
"pickle.dump( {'words':words, 'classes':classes, 'train_x':train_x, 'train_y':train_y}, open( \"training_data\", \"wb\" ) )"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(\"Loading Pickle.....\")\n",
"data = pickle.load( open( \"training_data\", \"rb\" ) )\n",
"words = data['words']\n",
"classes = data['classes']\n",
"train_x = data['train_x']\n",
"train_y = data['train_y']\n",
"\n",
"\n",
"with open(train_data_file) as json_data:\n",
" intents = json.load(json_data)\n",
" \n",
"print(\"Loading the Model......\")\n",
"# load our saved model\n",
"model.load('./model.tflearn')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def clean_up_sentence(sentence):\n",
" # It Tokenize or Break it into the constituents parts of Sentense.\n",
" sentence_words = nltk.word_tokenize(sentence)\n",
" # Stemming means to find the root of the word.\n",
" sentence_words = [stemmer.stem(word.lower()) for word in sentence_words]\n",
" return sentence_words\n",
"\n",
"# Return the Array of Bag of Words: True or False and 0 or 1 for each word of bag that exists in the Sentence\n",
"def bow(sentence, words, show_details=False):\n",
" sentence_words = clean_up_sentence(sentence)\n",
" bag = [0]*len(words)\n",
" for s in sentence_words:\n",
" for i,w in enumerate(words):\n",
" if w == s:\n",
" bag[i] = 1\n",
" if show_details:\n",
" print (\"found in bag: %s\" % w)\n",
" return(np.array(bag))\n",
"\n",
"ERROR_THRESHOLD = 0.25\n",
"print(\"ERROR_THRESHOLD = 0.25\")\n",
"\n",
"def classify(sentence):\n",
" # Prediction or To Get the Posibility or Probability from the Model\n",
" results = model.predict([bow(sentence, words)])[0]\n",
" # Exclude those results which are Below Threshold\n",
" results = [[i,r] for i,r in enumerate(results) if r>ERROR_THRESHOLD]\n",
" # Sorting is Done because heigher Confidence Answer comes first.\n",
" results.sort(key=lambda x: x[1], reverse=True)\n",
" return_list = []\n",
" for r in results:\n",
" return_list.append((classes[r[0]], r[1])) #Tuppl -> Intent and Probability\n",
" return return_list\n",
"\n",
"def response(sentence, userID='123', show_details=False):\n",
" results = classify(sentence)\n",
" # That Means if Classification is Done then Find the Matching Tag.\n",
" if results:\n",
" # Long Loop to get the Result.\n",
" while results:\n",
" for i in intents['intents']:\n",
" # Tag Finding\n",
" if i['tag'] == results[0][0]:\n",
" # Random Response from High Order Probabilities\n",
" return print(random.choice(i['responses']))\n",
"\n",
" results.pop(0)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 💬 Testing"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"while True:\n",
" input_data = input(\"You- \")\n",
" if(input_data=='q'):\n",
" break;\n",
" answer = response(input_data)\n",
" answer"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"provenance": []
},
"gpuClass": "standard",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.0"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
BackEnd/Classify_Questions.ipynb 0 → 100644
View file @ 5d53e813
{
"cells": [
{
"cell_type": "code",
"execution_count": 40,
"metadata": {
"id": "fUbFvR7LDB3U"
},
"outputs": [],
"source": [
"#Importing required Dependencies\n",
"from __future__ import absolute_import, division, print_function, unicode_literals\n",
"import tensorflow as tf\n",
"\n",
"import os\n",
"import datetime\n",
"import tensorflow_hub as hub\n",
"import numpy as np\n",
"import pandas as pd #alias\n",
"from sklearn.model_selection import train_test_split"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "7AvC6YiXfM86",
"outputId": "688daefe-24e4-4a6e-c871-f34926a5f2e4"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Mounted at /content/drive\n"
]
}
],
"source": [
"#Connecting this colab notebook to the Google Drive \n",
"from google.colab import drive\n",
"drive.mount('/content/drive')"
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {
"id": "QDH43tw-DN4y"
},
"outputs": [],
"source": [
"#Loading the Dataset into a pandas dataframe(a table like structure to keep the data temporary)\n",
"df2=pd.read_excel('/content/drive/MyDrive/Final Year Research/qClassifier_dataset.xlsx')"
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "AewvMzr1NBJc",
"outputId": "066f2176-36e7-4337-f419-5fa94545cde8"
},
"outputs": [
{
"data": {
"text/plain": [
"Index(['text', 'target'], dtype='object')"
]
},
"execution_count": 42,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#check the columns of the dataframe\n",
"df2.columns"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 206
},
"id": "y-MwPIj8LvRX",
"outputId": "4dfe95b8-dbcd-4047-816d-8eea22329541"
},
"outputs": [
{
"data": {
"text/html": [
"\n",
" <div id=\"df-60a78dc3-7275-4c1a-b2cd-c17dcbcbf4b2\">\n",
" <div class=\"colab-df-container\">\n",
" <div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>text</th>\n",
" <th>target</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>Hi</td>\n",
" <td>greeting</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>How are you</td>\n",
" <td>greeting</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>Is anyone there?</td>\n",
" <td>greeting</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>Hello</td>\n",
" <td>greeting</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>Good day</td>\n",
" <td>greeting</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>\n",
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-60a78dc3-7275-4c1a-b2cd-c17dcbcbf4b2')\"\n",
" title=\"Convert this dataframe to an interactive table.\"\n",
" style=\"display:none;\">\n",
" \n",
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
" width=\"24px\">\n",
" <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
" <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
" </svg>\n",
" </button>\n",
" \n",
" <style>\n",
" .colab-df-container {\n",
" display:flex;\n",
" flex-wrap:wrap;\n",
" gap: 12px;\n",
" }\n",
"\n",
" .colab-df-convert {\n",
" background-color: #E8F0FE;\n",
" border: none;\n",
" border-radius: 50%;\n",
" cursor: pointer;\n",
" display: none;\n",
" fill: #1967D2;\n",
" height: 32px;\n",
" padding: 0 0 0 0;\n",
" width: 32px;\n",
" }\n",
"\n",
" .colab-df-convert:hover {\n",
" background-color: #E2EBFA;\n",
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
" fill: #174EA6;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-convert {\n",
" background-color: #3B4455;\n",
" fill: #D2E3FC;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-convert:hover {\n",
" background-color: #434B5C;\n",
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
" fill: #FFFFFF;\n",
" }\n",
" </style>\n",
"\n",
" <script>\n",
" const buttonEl =\n",
" document.querySelector('#df-60a78dc3-7275-4c1a-b2cd-c17dcbcbf4b2 button.colab-df-convert');\n",
" buttonEl.style.display =\n",
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
"\n",
" async function convertToInteractive(key) {\n",
" const element = document.querySelector('#df-60a78dc3-7275-4c1a-b2cd-c17dcbcbf4b2');\n",
" const dataTable =\n",
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
" [key], {});\n",
" if (!dataTable) return;\n",
"\n",
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
" + ' to learn more about interactive tables.';\n",
" element.innerHTML = '';\n",
" dataTable['output_type'] = 'display_data';\n",
" await google.colab.output.renderOutput(dataTable, element);\n",
" const docLink = document.createElement('div');\n",
" docLink.innerHTML = docLinkHtml;\n",
" element.appendChild(docLink);\n",
" }\n",
" </script>\n",
" </div>\n",
" </div>\n",
" "
],
"text/plain": [
" text target\n",
"0 Hi greeting\n",
"1 How are you greeting\n",
"2 Is anyone there? greeting\n",
"3 Hello greeting\n",
"4 Good day greeting"
]
},
"execution_count": 43,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df2.head() #checking the top 5 rows , use df2.tail() if u need to see the last 5 rows"
]
},
{
"cell_type": "code",
"execution_count": 44,
"metadata": {
"id": "U_UepjKp0fJ-"
},
"outputs": [],
"source": [
"X_train, X_test = train_test_split(df2, test_size=0.2, random_state=42)#random state is used to re-produce the same dataset"
]
},
{
"cell_type": "code",
"execution_count": 45,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"id": "LlSDPQOXPVzH",
"outputId": "90e86271-b3b9-48de-8ca9-017bf55f06d3"
},
"outputs": [
{
"data": {
"text/html": [
"\n",
" <div id=\"df-3d6f5791-42cb-4678-ac72-e4a196c9b030\">\n",
" <div class=\"colab-df-container\">\n",
" <div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>text</th>\n",
" <th>target</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>Hello</td>\n",
" <td>greeting</td>\n",
" </tr>\n",
" <tr>\n",
" <th>6</th>\n",
" <td>Compost production is a very useful process fo...</td>\n",
" <td>compost</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>During water supply, sludge/fluid chemical com...</td>\n",
" <td>dripirrigation</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>a plant disease that can infect many common ve...</td>\n",
" <td>plantdiseases</td>\n",
" </tr>\n",
" <tr>\n",
" <th>19</th>\n",
" <td>Surface Drip Irrigation</td>\n",
" <td>dripirrigation</td>\n",
" </tr>\n",
" <tr>\n",
" <th>17</th>\n",
" <td>The basic 'philosohy of drip irrigation is Phi...</td>\n",
" <td>dripirrigation</td>\n",
" </tr>\n",
" <tr>\n",
" <th>13</th>\n",
" <td>It is easy to absorb nutrients in compost whic...</td>\n",
" <td>compost</td>\n",
" </tr>\n",
" <tr>\n",
" <th>40</th>\n",
" <td>Damage to garlic</td>\n",
" <td>plantdiseases</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>this process is very useful in urban environme...</td>\n",
" <td>compost</td>\n",
" </tr>\n",
" <tr>\n",
" <th>9</th>\n",
" <td>Helps to keep environment clean</td>\n",
" <td>compost</td>\n",
" </tr>\n",
" <tr>\n",
" <th>16</th>\n",
" <td>The water flowing through a lateral pipe syste...</td>\n",
" <td>dripirrigation</td>\n",
" </tr>\n",
" <tr>\n",
" <th>30</th>\n",
" <td>Quick facts</td>\n",
" <td>plantdiseases</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>Compliant with wastewater supply25% - 35% wate...</td>\n",
" <td>dripirrigation</td>\n",
" </tr>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>Hi</td>\n",
" <td>greeting</td>\n",
" </tr>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>Aster yellows</td>\n",
" <td>plantdiseases</td>\n",
" </tr>\n",
" <tr>\n",
" <th>41</th>\n",
" <td>Small, often soft bulbs</td>\n",
" <td>plantdiseases</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>Production of compost mean decomposition of pl...</td>\n",
" <td>compost</td>\n",
" </tr>\n",
" <tr>\n",
" <th>11</th>\n",
" <td>Increasing the fertility of the soil</td>\n",
" <td>compost</td>\n",
" </tr>\n",
" <tr>\n",
" <th>32</th>\n",
" <td>Infected plants have yellow leaves and stem, s...</td>\n",
" <td>plantdiseases</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>How are you</td>\n",
" <td>greeting</td>\n",
" </tr>\n",
" <tr>\n",
" <th>42</th>\n",
" <td>Dark streaking or discoloration of the wrapper</td>\n",
" <td>plantdiseases</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>Constant supply of water as per crop requireme...</td>\n",
" <td>dripirrigation</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>Is anyone there?</td>\n",
" <td>greeting</td>\n",
" </tr>\n",
" <tr>\n",
" <th>33</th>\n",
" <td>The insect aster leafhopper can carry the aste...</td>\n",
" <td>plantdiseases</td>\n",
" </tr>\n",
" <tr>\n",
" <th>23</th>\n",
" <td>a method of irrigation for agriculture as well...</td>\n",
" <td>dripirrigation</td>\n",
" </tr>\n",
" <tr>\n",
" <th>35</th>\n",
" <td>Plants infected with aster yellows should be r...</td>\n",
" <td>plantdiseases</td>\n",
" </tr>\n",
" <tr>\n",
" <th>10</th>\n",
" <td>Receiving a commercial value for the waste matter</td>\n",
" <td>compost</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>Dispersal irrigation (Sprinkler Irrigation</td>\n",
" <td>dripirrigation</td>\n",
" </tr>\n",
" <tr>\n",
" <th>18</th>\n",
" <td>There are two types of Drip Irrigation</td>\n",
" <td>dripirrigation</td>\n",
" </tr>\n",
" <tr>\n",
" <th>20</th>\n",
" <td>Major lakhs of drip irrigation</td>\n",
" <td>dripirrigation</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>Advantages of compost production</td>\n",
" <td>compost</td>\n",
" </tr>\n",
" <tr>\n",
" <th>14</th>\n",
" <td>Plant give quick response for compost than che...</td>\n",
" <td>compost</td>\n",
" </tr>\n",
" <tr>\n",
" <th>28</th>\n",
" <td>Leaf spots, fruit rot, wilt, and unusual plant...</td>\n",
" <td>plantdiseases</td>\n",
" </tr>\n",
" <tr>\n",
" <th>38</th>\n",
" <td>Leaves may be small and stunted</td>\n",
" <td>plantdiseases</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>\n",
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-3d6f5791-42cb-4678-ac72-e4a196c9b030')\"\n",
" title=\"Convert this dataframe to an interactive table.\"\n",
" style=\"display:none;\">\n",
" \n",
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
" width=\"24px\">\n",
" <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
" <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
" </svg>\n",
" </button>\n",
" \n",
" <style>\n",
" .colab-df-container {\n",
" display:flex;\n",
" flex-wrap:wrap;\n",
" gap: 12px;\n",
" }\n",
"\n",
" .colab-df-convert {\n",
" background-color: #E8F0FE;\n",
" border: none;\n",
" border-radius: 50%;\n",
" cursor: pointer;\n",
" display: none;\n",
" fill: #1967D2;\n",
" height: 32px;\n",
" padding: 0 0 0 0;\n",
" width: 32px;\n",
" }\n",
"\n",
" .colab-df-convert:hover {\n",
" background-color: #E2EBFA;\n",
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
" fill: #174EA6;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-convert {\n",
" background-color: #3B4455;\n",
" fill: #D2E3FC;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-convert:hover {\n",
" background-color: #434B5C;\n",
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
" fill: #FFFFFF;\n",
" }\n",
" </style>\n",
"\n",
" <script>\n",
" const buttonEl =\n",
" document.querySelector('#df-3d6f5791-42cb-4678-ac72-e4a196c9b030 button.colab-df-convert');\n",
" buttonEl.style.display =\n",
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
"\n",
" async function convertToInteractive(key) {\n",
" const element = document.querySelector('#df-3d6f5791-42cb-4678-ac72-e4a196c9b030');\n",
" const dataTable =\n",
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
" [key], {});\n",
" if (!dataTable) return;\n",
"\n",
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
" + ' to learn more about interactive tables.';\n",
" element.innerHTML = '';\n",
" dataTable['output_type'] = 'display_data';\n",
" await google.colab.output.renderOutput(dataTable, element);\n",
" const docLink = document.createElement('div');\n",
" docLink.innerHTML = docLinkHtml;\n",
" element.appendChild(docLink);\n",
" }\n",
" </script>\n",
" </div>\n",
" </div>\n",
" "
],
"text/plain": [
" text target\n",
"3 Hello greeting\n",
"6 Compost production is a very useful process fo... compost\n",
"27 During water supply, sludge/fluid chemical com... dripirrigation\n",
"31 a plant disease that can infect many common ve... plantdiseases\n",
"19 Surface Drip Irrigation dripirrigation\n",
"17 The basic 'philosohy of drip irrigation is Phi... dripirrigation\n",
"13 It is easy to absorb nutrients in compost whic... compost\n",
"40 Damage to garlic plantdiseases\n",
"15 this process is very useful in urban environme... compost\n",
"9 Helps to keep environment clean compost\n",
"16 The water flowing through a lateral pipe syste... dripirrigation\n",
"30 Quick facts plantdiseases\n",
"26 Compliant with wastewater supply25% - 35% wate... dripirrigation\n",
"0 Hi greeting\n",
"29 Aster yellows plantdiseases\n",
"41 Small, often soft bulbs plantdiseases\n",
"5 Production of compost mean decomposition of pl... compost\n",
"11 Increasing the fertility of the soil compost\n",
"32 Infected plants have yellow leaves and stem, s... plantdiseases\n",
"1 How are you greeting\n",
"42 Dark streaking or discoloration of the wrapper plantdiseases\n",
"21 Constant supply of water as per crop requireme... dripirrigation\n",
"2 Is anyone there? greeting\n",
"33 The insect aster leafhopper can carry the aste... plantdiseases\n",
"23 a method of irrigation for agriculture as well... dripirrigation\n",
"35 Plants infected with aster yellows should be r... plantdiseases\n",
"10 Receiving a commercial value for the waste matter compost\n",
"22 Dispersal irrigation (Sprinkler Irrigation dripirrigation\n",
"18 There are two types of Drip Irrigation dripirrigation\n",
"20 Major lakhs of drip irrigation dripirrigation\n",
"7 Advantages of compost production compost\n",
"14 Plant give quick response for compost than che... compost\n",
"28 Leaf spots, fruit rot, wilt, and unusual plant... plantdiseases\n",
"38 Leaves may be small and stunted plantdiseases"
]
},
"execution_count": 45,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"X_train"
]
},
{
"cell_type": "code",
"execution_count": 46,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "Hr2mv9PUjNQ-",
"outputId": "983bc651-c63a-4678-dbe5-fa85c72692df"
},
"outputs": [
{
"data": {
"text/plain": [
"plantdiseases 15\n",
"dripirrigation 12\n",
"compost 11\n",
"greeting 5\n",
"Name: target, dtype: int64"
]
},
"execution_count": 46,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df2['target'].value_counts()#take the unique targets/classes and count the number of rows for each of them"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "WqODvBFKOfzu",
"outputId": "95092230-2e78-41f1-cbe1-15bac63f35ce"
},
"outputs": [
{
"data": {
"text/plain": [
"array(['compost', 'dripirrigation', 'greeting', 'plantdiseases'],\n",
" dtype=object)"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": []
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {
"id": "J6tvBkIrgRk2"
},
"outputs": [],
"source": [
"from sklearn.utils.class_weight import compute_class_weight\n",
"#Assigning weights to the target labels\n",
"class_weights = compute_class_weight(\n",
" class_weight = \"balanced\",\n",
" classes = np.unique(df2['target']),\n",
" y = df2['target'] \n",
" )\n",
"class_weights.sort()"
]
},
{
"cell_type": "code",
"execution_count": 48,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "FrHpKpx1OpMe",
"outputId": "d86eda26-fa15-4e36-bed9-57dcbf1dd691"
},
"outputs": [
{
"data": {
"text/plain": [
"array([0.71666667, 0.89583333, 0.97727273, 2.15 ])"
]
},
"execution_count": 48,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"class_weights"
]
},
{
"cell_type": "code",
"execution_count": 49,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "8brlwCEg0vnf",
"outputId": "001356d5-2e9b-4577-ca7b-b1b3a934fd95"
},
"outputs": [
{
"data": {
"text/plain": [
"{0: 0.7166666666666667, 1: 0.8958333333333334, 2: 0.9772727272727273, 3: 2.15}"
]
},
"execution_count": 49,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#Since keras needs weights as a dict object, converting the weight list to a dict\n",
"weights={}\n",
"for index, weight in enumerate(class_weights) :\n",
" weights[index]=weight\n",
"weights #A dict is a set of key value pairs enclosed with a {}"
]
},
{
"cell_type": "code",
"execution_count": 50,
"metadata": {
"id": "hijEnYw7V_ey"
},
"outputs": [],
"source": [
"#convet the dataframe/list to a tensorflow/keras dataset\n",
"dataset_train = tf.data.Dataset.from_tensor_slices((X_train['text'].values, X_train['target'].values))\n",
"dataset_test = tf.data.Dataset.from_tensor_slices((X_test['text'].values, X_test['target'].values))"
]
},
{
"cell_type": "code",
"execution_count": 51,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "u1diITDtD9xD",
"outputId": "d0c8b007-341f-4b6b-f1fd-7c817e8375d8"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Question: b'Hello', Type: b'greeting'\n",
"Question: b'Compost production is a very useful process for the environment,society and also for the agriculture industry.', Type: b'compost'\n",
"Question: b'During water supply, sludge/fluid chemical composition is formed.', Type: b'dripirrigation'\n",
"Question: b'a plant disease that can infect many common vegetables', Type: b'plantdiseases'\n",
"Question: b'Surface Drip Irrigation', Type: b'dripirrigation'\n",
"Question: b\"The basic 'philosohy of drip irrigation is Philosophy of Drip Irrigation is to irrigate the Plant root Zone but mot to the Soil\", Type: b'dripirrigation'\n",
"Question: b'It is easy to absorb nutrients in compost which are good for the plant', Type: b'compost'\n",
"Question: b'Damage to garlic', Type: b'plantdiseases'\n"
]
}
],
"source": [
"for text, target in dataset_train.take(8):\n",
" print ('Question: {}, Type: {}'.format(text, target))"
]
},
{
"cell_type": "code",
"execution_count": 52,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "1gVAVaNeoNEH",
"outputId": "87577234-b6da-45ea-b627-93790e6e460e"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Question: b'Leaves are discoloured from pale green to yellow or white. In some plants, red to purple discoloration of leaves occurs', Type: b'plantdiseases'\n",
"Question: b'Dispersal water supply', Type: b'dripirrigation'\n",
"Question: b'Upwelling tube ends attached', Type: b'dripirrigation'\n",
"Question: b'Plants infected early in the growing season may remain small and stunted', Type: b'plantdiseases'\n",
"Question: b'Once infected with aster yellows, a plant will never recover', Type: b'plantdiseases'\n"
]
}
],
"source": [
"for text, target in dataset_test.take(5):\n",
" print ('Question: {}, Type: {}'.format(text, target))"
]
},
{
"cell_type": "code",
"execution_count": 53,
"metadata": {
"id": "posXzJ1PvHZl"
},
"outputs": [],
"source": [
"#since we have the targets as text values like , greeting, compost... \n",
"#model is looking for only numeric values , therefore we need to convert this classes into numbers, ex- greeting -0 , compost-1\n",
"\n",
"table = tf.lookup.StaticHashTable(\n",
" initializer=tf.lookup.KeyValueTensorInitializer(\n",
" keys=tf.constant(['plantdiseases', 'dripirrigation','compost','greeting']),\n",
" values=tf.constant([0, 1, 2, 3]),\n",
" ),\n",
" default_value=tf.constant(-1),\n",
" name=\"target_encoding\"\n",
")\n",
"\n",
"@tf.function\n",
"def target(x):\n",
" return table.lookup(x)"
]
},
{
"cell_type": "code",
"execution_count": 54,
"metadata": {
"id": "dDr8nLr8vIe-"
},
"outputs": [],
"source": [
"def show_batch(dataset, size=5):\n",
" for batch, label in dataset.take(size):\n",
" print(batch.numpy())\n",
" print(target(label).numpy())"
]
},
{
"cell_type": "code",
"execution_count": 55,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "YbJgzzAmIqTp",
"outputId": "47360f7c-1eb3-469a-c86c-4ea2e3c65f55"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"b'Leaves are discoloured from pale green to yellow or white. In some plants, red to purple discoloration of leaves occurs'\n",
"0\n",
"b'Dispersal water supply'\n",
"1\n",
"b'Upwelling tube ends attached'\n",
"1\n",
"b'Plants infected early in the growing season may remain small and stunted'\n",
"0\n",
"b'Once infected with aster yellows, a plant will never recover'\n",
"0\n",
"b'Flowers are small, malformed, and often remain green or fail to develop the proper colour'\n",
"0\n"
]
}
],
"source": [
"show_batch(dataset_test,6)"
]
},
{
"cell_type": "code",
"execution_count": 56,
"metadata": {
"id": "d6D4lGAezmjL"
},
"outputs": [],
"source": [
"#this is just a function\n",
"def fetch(text, labels):\n",
" return text, tf.one_hot(target(labels),4) # 4 - means the no of classes"
]
},
{
"cell_type": "code",
"execution_count": 57,
"metadata": {
"id": "DQX8hZl60WaK"
},
"outputs": [],
"source": [
"train_data_f=dataset_train.map(fetch) #this function will executed on each row of the dataset\n",
"test_data_f=dataset_test.map(fetch)"
]
},
{
"cell_type": "code",
"execution_count": 58,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "xOzA_Tz4vIcV",
"outputId": "4c656f5f-2bc5-45fb-a89d-48e85bc76206"
},
"outputs": [
{
"data": {
"text/plain": [
"(<tf.Tensor: shape=(), dtype=string, numpy=b'Hello'>,\n",
" <tf.Tensor: shape=(4,), dtype=float32, numpy=array([0., 0., 0., 1.], dtype=float32)>)"
]
},
"execution_count": 58,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"next(iter(train_data_f))"
]
},
{
"cell_type": "code",
"execution_count": 59,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "Zad9I-x19Vvo",
"outputId": "c733572f-660e-4c66-fbbb-b8a97d002252"
},
"outputs": [
{
"data": {
"text/plain": [
"(<tf.Tensor: shape=(5,), dtype=string, numpy=\n",
" array([b'Hello',\n",
" b'Compost production is a very useful process for the environment,society and also for the agriculture industry.',\n",
" b'During water supply, sludge/fluid chemical composition is formed.',\n",
" b'a plant disease that can infect many common vegetables',\n",
" b'Surface Drip Irrigation'], dtype=object)>,\n",
" <tf.Tensor: shape=(5, 4), dtype=float32, numpy=\n",
" array([[0., 0., 0., 1.],\n",
" [0., 0., 1., 0.],\n",
" [0., 1., 0., 0.],\n",
" [1., 0., 0., 0.],\n",
" [0., 1., 0., 0.]], dtype=float32)>)"
]
},
"execution_count": 59,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train_data, train_labels = next(iter(train_data_f.batch(5)))\n",
"train_data, train_labels"
]
},
{
"cell_type": "code",
"execution_count": 60,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "ifU5cdUp9VnZ",
"outputId": "f5a1c497-164d-4c7c-e7c5-dbbe2e027838"
},
"outputs": [
{
"data": {
"text/plain": [
"<tf.Tensor: shape=(1, 128), dtype=float32, numpy=\n",
"array([[-0.01326104, -0.06651955, 0.05000011, 0.00160342, 0.0924684 ,\n",
" 0.04142092, 0.07584989, 0.01886188, 0.03389315, -0.12547898,\n",
" -0.19535598, -0.11637609, -0.10395012, -0.06446726, 0.15113464,\n",
" 0.09859799, -0.07753015, -0.09837861, -0.06968897, 0.07707419,\n",
" -0.18155406, -0.09807127, 0.1477701 , 0.05832554, 0.15936404,\n",
" -0.12575397, -0.05127091, -0.00821524, -0.08168326, 0.07887778,\n",
" -0.0494835 , -0.01654673, -0.08092603, -0.11115438, 0.07077679,\n",
" 0.10004672, -0.14759016, -0.03613551, 0.05240625, 0.13715076,\n",
" 0.02053809, -0.10842169, 0.02835904, -0.09347737, 0.0504682 ,\n",
" -0.04231362, 0.06257773, -0.10890798, -0.04440736, -0.04152 ,\n",
" 0.01575615, 0.06000478, -0.03723849, 0.20751607, -0.08417128,\n",
" 0.04031491, -0.02655242, -0.04685495, 0.07308991, -0.10720447,\n",
" -0.01765174, 0.00542998, 0.15110734, -0.06488985, 0.12080824,\n",
" 0.081383 , -0.01847771, -0.0362265 , 0.0058647 , 0.02230326,\n",
" 0.02772313, 0.17580664, -0.06658121, 0.09473604, -0.03688768,\n",
" -0.01979401, -0.02077567, -0.02271776, 0.02573858, 0.05813851,\n",
" -0.02174116, 0.08787552, -0.08649857, 0.04052519, 0.07276539,\n",
" 0.0169026 , -0.06583107, -0.0418334 , 0.01385752, -0.01216413,\n",
" 0.05700318, -0.04536071, -0.00724975, -0.01004006, 0.11074897,\n",
" 0.00502053, -0.10549591, 0.04251076, 0.06499296, -0.01479268,\n",
" -0.10494796, 0.12079915, 0.1513217 , -0.16735281, -0.18061689,\n",
" 0.08851749, -0.08837191, 0.04744839, -0.10761594, -0.13394493,\n",
" 0.1500802 , -0.01432055, 0.00259216, 0.2582441 , -0.00921307,\n",
" 0.1534771 , 0.05704968, -0.03371219, -0.02885139, -0.10249937,\n",
" -0.01767499, -0.07923365, 0.06045366, 0.12972409, 0.00327356,\n",
" -0.04011271, 0.14959796, -0.04728057]], dtype=float32)>"
]
},
"execution_count": 60,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#This is a transfer-learning/using a pre=trained models\n",
"embedding = \"https://tfhub.dev/google/tf2-preview/nnlm-en-dim128/1\"\n",
"hub_layer = hub.KerasLayer(embedding, output_shape=[128], input_shape=[], \n",
" dtype=tf.string, trainable=True)\n",
"hub_layer(train_data[:1])"
]
},
{
"cell_type": "code",
"execution_count": 61,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "1AI7BhDe9Vif",
"outputId": "57ee3a00-76c6-4514-d515-d6ce4bb23814"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model: \"sequential_1\"\n",
"_________________________________________________________________\n",
" Layer (type) Output Shape Param # \n",
"=================================================================\n",
" keras_layer_1 (KerasLayer) (None, 128) 124642688 \n",
" \n",
" dense_5 (Dense) (None, 128) 16512 \n",
" \n",
" dropout_4 (Dropout) (None, 128) 0 \n",
" \n",
" dense_6 (Dense) (None, 128) 16512 \n",
" \n",
" dropout_5 (Dropout) (None, 128) 0 \n",
" \n",
" dense_7 (Dense) (None, 64) 8256 \n",
" \n",
" dropout_6 (Dropout) (None, 64) 0 \n",
" \n",
" dense_8 (Dense) (None, 32) 2080 \n",
" \n",
" dropout_7 (Dropout) (None, 32) 0 \n",
" \n",
" dense_9 (Dense) (None, 4) 132 \n",
" \n",
"=================================================================\n",
"Total params: 124,686,180\n",
"Trainable params: 124,686,180\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
]
}
],
"source": [
"#01- create the model\n",
"model = tf.keras.Sequential()\n",
"model.add(hub_layer)\n",
"for units in [128, 128, 64 , 32]:\n",
" model.add(tf.keras.layers.Dense(units, activation='relu'))\n",
" model.add(tf.keras.layers.Dropout(0.3))\n",
"model.add(tf.keras.layers.Dense(4, activation='softmax')) #this isthe output layer ,4 is the number of classes if it is more then 2 - actiation - softmax else (2) sigmoid\n",
"\n",
"model.summary()"
]
},
{
"cell_type": "code",
"execution_count": 62,
"metadata": {
"id": "Z_HPHq3W9VeT"
},
"outputs": [],
"source": [
"#02 - compile the model\n",
"model.compile(optimizer='adam',\n",
" loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True), #BinaryCrossentropy\n",
" metrics=['accuracy'])"
]
},
{
"cell_type": "code",
"execution_count": 63,
"metadata": {
"id": "jSCeGUYF9TlL"
},
"outputs": [],
"source": [
"#to avoid the patterns we suffle the dataset\n",
"train_data_f=train_data_f.shuffle(70000).batch(512)\n",
"test_data_f=test_data_f.batch(512)"
]
},
{
"cell_type": "code",
"execution_count": 64,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "E_h5I4StQj5E",
"outputId": "75949725-a7e0-41f2-fc84-71ebb6fab159"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/4\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/usr/local/lib/python3.9/dist-packages/keras/backend.py:5561: UserWarning: \"`categorical_crossentropy` received `from_logits=True`, but the `output` argument was produced by a Softmax activation and thus does not represent logits. Was this intended?\n",
" output, from_logits = _get_logits(\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"1/1 [==============================] - 7s 7s/step - loss: 1.4121 - accuracy: 0.1471 - val_loss: 1.3852 - val_accuracy: 0.1111\n",
"Epoch 2/4\n",
"1/1 [==============================] - 0s 83ms/step - loss: 1.4214 - accuracy: 0.0588 - val_loss: 1.3797 - val_accuracy: 0.1111\n",
"Epoch 3/4\n",
"1/1 [==============================] - 0s 95ms/step - loss: 1.3827 - accuracy: 0.3235 - val_loss: 1.3755 - val_accuracy: 0.4444\n",
"Epoch 4/4\n",
"1/1 [==============================] - 0s 96ms/step - loss: 1.3761 - accuracy: 0.3529 - val_loss: 1.3719 - val_accuracy: 0.5556\n"
]
}
],
"source": [
"#03 - fitting / training the model\n",
"history = model.fit(train_data_f,\n",
" epochs=4,\n",
" validation_data=test_data_f,\n",
" verbose=1\n",
" #class_weight=weights\n",
" )"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "hy55ceBpjL7E",
"outputId": "48c0a850-2246-43c3-cecd-76d300109103"
},
"outputs": [
{
"data": {
"text/plain": [
"9"
]
},
"execution_count": 53,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(list(dataset_test))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "MwG1bP-TC5fE",
"outputId": "6c288773-fadc-4a89-c697-d0e64ff3dd4d"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1/1 - 0s - loss: 1.3658 - accuracy: 0.2222 - 45ms/epoch - 45ms/step\n",
"[1.3657673597335815, 0.2222222238779068]\n"
]
}
],
"source": [
"results = model.evaluate(dataset_test.map(fetch).batch(11491), verbose=2)\n",
"\n",
"print(results)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-nQdChwqbFer"
},
"outputs": [],
"source": [
"test_data, test_labels = next(iter(dataset_test.map(fetch).batch(45963)))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "fIrFsufOl1s2",
"outputId": "854f7886-4be2-447b-d469-ef1f6c7590f6"
},
"outputs": [
{
"data": {
"text/plain": [
"<tf.Tensor: shape=(9,), dtype=string, numpy=\n",
"array([b'Hi', b'Helps to keep environment clean',\n",
" b'Once infected with aster yellows, a plant will never recover',\n",
" b'Infected plants have yellow leaves and stem, stunted growth, and small malformed flowers',\n",
" b'Quick facts', b'Leaves may be small and stunted',\n",
" b'a plant disease that can infect many common vegetables',\n",
" b\"The basic 'philosohy of drip irrigation is Philosophy of Drip Irrigation is to irrigate the Plant root Zone but mot to the Soil\",\n",
" b'Hello'], dtype=object)>"
]
},
"execution_count": 62,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"test_data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "nZ9qnGJ7cHlc",
"outputId": "999eb530-5620-430f-bf37-89d3e2ed02ec"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1/1 [==============================] - 0s 117ms/step\n"
]
}
],
"source": [
"y_pred=model.predict(test_data)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "ZzzDvHTAcnIv",
"outputId": "f509af7d-4d46-4bd1-ebca-303df68daf59"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" precision recall f1-score support\n",
"\n",
" 0 0.00 0.00 0.00 5\n",
" 1 0.25 1.00 0.40 1\n",
" 2 0.20 1.00 0.33 1\n",
" 3 0.00 0.00 0.00 2\n",
"\n",
" accuracy 0.22 9\n",
" macro avg 0.11 0.50 0.18 9\n",
"weighted avg 0.05 0.22 0.08 9\n",
"\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/usr/local/lib/python3.9/dist-packages/sklearn/metrics/_classification.py:1344: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
" _warn_prf(average, modifier, msg_start, len(result))\n",
"/usr/local/lib/python3.9/dist-packages/sklearn/metrics/_classification.py:1344: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
" _warn_prf(average, modifier, msg_start, len(result))\n",
"/usr/local/lib/python3.9/dist-packages/sklearn/metrics/_classification.py:1344: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
" _warn_prf(average, modifier, msg_start, len(result))\n"
]
}
],
"source": [
"from sklearn.metrics import classification_report\n",
"print(classification_report(test_labels.numpy().argmax(axis=1), y_pred.argmax(axis=1)))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "KGQ59jwIQjgX",
"outputId": "95463afe-6017-4236-e1f4-4b309f486469"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[0, 3, 2, 0],\n",
" [0, 1, 0, 0],\n",
" [0, 0, 1, 0],\n",
" [0, 0, 2, 0]])"
]
},
"execution_count": 58,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from sklearn.metrics import confusion_matrix\n",
"confusion_matrix(test_labels.numpy().argmax(axis=1), y_pred.argmax(axis=1))"
]
},
{
"cell_type": "code",
"execution_count": 65,
"metadata": {
"id": "xM7snwiBpO4v"
},
"outputs": [],
"source": [
"model.save('qclassifiertf.h5')"
]
},
{
"cell_type": "code",
"execution_count": 66,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "JzfPjub9pOxl",
"outputId": "ef980911-c5d3-4b9f-c877-5e61d13b7b05"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model: \"sequential_1\"\n",
"_________________________________________________________________\n",
" Layer (type) Output Shape Param # \n",
"=================================================================\n",
" keras_layer_1 (KerasLayer) (None, 128) 124642688 \n",
" \n",
" dense_5 (Dense) (None, 128) 16512 \n",
" \n",
" dropout_4 (Dropout) (None, 128) 0 \n",
" \n",
" dense_6 (Dense) (None, 128) 16512 \n",
" \n",
" dropout_5 (Dropout) (None, 128) 0 \n",
" \n",
" dense_7 (Dense) (None, 64) 8256 \n",
" \n",
" dropout_6 (Dropout) (None, 64) 0 \n",
" \n",
" dense_8 (Dense) (None, 32) 2080 \n",
" \n",
" dropout_7 (Dropout) (None, 32) 0 \n",
" \n",
" dense_9 (Dense) (None, 4) 132 \n",
" \n",
"=================================================================\n",
"Total params: 124,686,180\n",
"Trainable params: 124,686,180\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
]
}
],
"source": [
"import tensorflow as tf\n",
"loaded = tf.keras.models.load_model('/content/qclassifiertf.h5' , custom_objects={'KerasLayer':hub.KerasLayer})\n",
"loaded.summary()"
]
},
{
"cell_type": "code",
"execution_count": 69,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "P_4cly4-leeP",
"outputId": "6e9cc6ca-93ba-405c-aff0-b3836b3133ce"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1/1 [==============================] - 0s 79ms/step\n"
]
}
],
"source": [
"res = loaded.predict(['How to make a compost at home'])"
]
},
{
"cell_type": "code",
"execution_count": 70,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "jcp7P1UGl7bv",
"outputId": "cec8d3db-a36c-4776-8fa7-272ab172cb62"
},
"outputs": [
{
"data": {
"text/plain": [
"2"
]
},
"execution_count": 70,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"np.argmax(res)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "fEgfnzU_ZFOu"
},
"outputs": [],
"source": []
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
BackEnd/IR_txtai.ipynb 0 → 100644
View file @ 5d53e813
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "JcaXE6lXyiO3",
"outputId": "940063cb-2ae6-4df1-b431-e11c737a2df9"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
"Collecting txtai\n",
" Downloading txtai-5.4.0-py3-none-any.whl (166 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m166.7/166.7 KB\u001b[0m \u001b[31m4.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting transformers>=4.22.0\n",
" Downloading transformers-4.27.4-py3-none-any.whl (6.8 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.8/6.8 MB\u001b[0m \u001b[31m44.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: numpy>=1.18.4 in /usr/local/lib/python3.9/dist-packages (from txtai) (1.22.4)\n",
"Requirement already satisfied: pyyaml>=5.3 in /usr/local/lib/python3.9/dist-packages (from txtai) (6.0)\n",
"Collecting faiss-cpu>=1.7.1.post2\n",
" Downloading faiss_cpu-1.7.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.0 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m17.0/17.0 MB\u001b[0m \u001b[31m48.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: torch>=1.6.0 in /usr/local/lib/python3.9/dist-packages (from txtai) (2.0.0+cu118)\n",
"Requirement already satisfied: networkx in /usr/local/lib/python3.9/dist-packages (from torch>=1.6.0->txtai) (3.0)\n",
"Requirement already satisfied: sympy in /usr/local/lib/python3.9/dist-packages (from torch>=1.6.0->txtai) (1.11.1)\n",
"Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.9/dist-packages (from torch>=1.6.0->txtai) (2.0.0)\n",
"Requirement already satisfied: filelock in /usr/local/lib/python3.9/dist-packages (from torch>=1.6.0->txtai) (3.10.7)\n",
"Requirement already satisfied: typing-extensions in /usr/local/lib/python3.9/dist-packages (from torch>=1.6.0->txtai) (4.5.0)\n",
"Requirement already satisfied: jinja2 in /usr/local/lib/python3.9/dist-packages (from torch>=1.6.0->txtai) (3.1.2)\n",
"Requirement already satisfied: lit in /usr/local/lib/python3.9/dist-packages (from triton==2.0.0->torch>=1.6.0->txtai) (16.0.0)\n",
"Requirement already satisfied: cmake in /usr/local/lib/python3.9/dist-packages (from triton==2.0.0->torch>=1.6.0->txtai) (3.25.2)\n",
"Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.9/dist-packages (from transformers>=4.22.0->txtai) (23.0)\n",
"Collecting tokenizers!=0.11.3,<0.14,>=0.11.1\n",
" Downloading tokenizers-0.13.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.8/7.8 MB\u001b[0m \u001b[31m52.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.9/dist-packages (from transformers>=4.22.0->txtai) (4.65.0)\n",
"Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.9/dist-packages (from transformers>=4.22.0->txtai) (2022.10.31)\n",
"Collecting huggingface-hub<1.0,>=0.11.0\n",
" Downloading huggingface_hub-0.13.4-py3-none-any.whl (200 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m200.1/200.1 KB\u001b[0m \u001b[31m13.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: requests in /usr/local/lib/python3.9/dist-packages (from transformers>=4.22.0->txtai) (2.27.1)\n",
"Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.9/dist-packages (from jinja2->torch>=1.6.0->txtai) (2.1.2)\n",
"Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests->transformers>=4.22.0->txtai) (3.4)\n",
"Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests->transformers>=4.22.0->txtai) (2.0.12)\n",
"Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests->transformers>=4.22.0->txtai) (1.26.15)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests->transformers>=4.22.0->txtai) (2022.12.7)\n",
"Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.9/dist-packages (from sympy->torch>=1.6.0->txtai) (1.3.0)\n",
"Installing collected packages: tokenizers, faiss-cpu, huggingface-hub, transformers, txtai\n",
"Successfully installed faiss-cpu-1.7.3 huggingface-hub-0.13.4 tokenizers-0.13.3 transformers-4.27.4 txtai-5.4.0\n"
]
}
],
"source": [
"!pip install txtai"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ttTEUck-xAVy"
},
"outputs": [],
"source": [
"from txtai.embeddings import Embeddings\n",
"import json"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Create embeddings model, backed by sentence-transformers & transformers\n",
"embeddings = Embeddings({\"path\": \"sentence-transformers/all-MiniLM-L6-v2\"})"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"with open(\"/content/sample.json\", \"r\") as f:\n",
" data = json.load(f)[\"descriptions\"][:100]\n",
"len(data)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"txtai_data = []\n",
"i = 0\n",
"for text in data:\n",
" txtai_data.append((i, text, None))\n",
" i=i+1\n",
"print(len(txtai_data))\n",
"txtai_data[1]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Create an index for the list of text\n",
"embeddings.index(txtai_data)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"res = embeddings.search(\"arson\", 10)\n",
"res"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"res = embeddings.search(\"what is explosive in a car\", 10)\n",
"for r in res:\n",
" print(f\"Text: {data[r[0]]}\")\n",
" print(f\"Similarity: {r[1]}\")\n",
" print()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"embeddings.save(\"models/test_model\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##Loading the Model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from txtai.embeddings import Embeddings\n",
"embeddings = Embeddings({\"path\": \"sentence-transformers/all-MiniLM-L6-v2\"})\n",
"\n",
"with open(\"/content/sample.json\", \"r\") as f:\n",
" data = json.load(f)[\"descriptions\"][:100]\n",
"len(data)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"embeddings.load(\"/content/models/test_model\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"res = embeddings.search(\"what is compost\", 3)\n",
"for r in res:\n",
" print(f\"Text: {data[r[0]]}\")\n",
" print(f\"Similarity: {r[1]}\")\n",
" print()"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
},
"widgets": {
"application/vnd.jupyter.widget-state+json": {
"02e7dd085a5548fe9276dacd6dac44d3": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"17ebec0847cd45068d51aea8b91b942a": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ProgressStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "ProgressStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"bar_color": null,
"description_width": ""
}
},
"19fe2169977f42d48fa87bba7c24f093": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"1b03824f9d814f799679cb9669b3f276": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"230429d7ebfe4b659c23ba1b7f292cc0": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"29c5530a3c6a4f2981eec6c83e1ed3b8": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "FloatProgressModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "FloatProgressModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "ProgressView",
"bar_style": "success",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_d3bb5e665cb04e25b35e250df0a025f0",
"max": 112,
"min": 0,
"orientation": "horizontal",
"style": "IPY_MODEL_3b17bfc82268496db611d51f12596f66",
"value": 112
}
},
"31569166f8614a4b98fa21095e7a085f": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "DescriptionStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"description_width": ""
}
},
"3177c9ac4640429e9ee7f69d78c21a4f": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HBoxModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HBoxModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HBoxView",
"box_style": "",
"children": [
"IPY_MODEL_f5e47bfbaef0488bbb0f1ef239facd32",
"IPY_MODEL_ba0d451eb6764795b95dfbf14aff5b0d",
"IPY_MODEL_aed8eedffd34480b81075da97630ae9c"
],
"layout": "IPY_MODEL_72f2f71bcd72468689d844da06a446cf"
}
},
"32635200eb334b33854bfb92a887a0e7": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "DescriptionStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"description_width": ""
}
},
"37e04779c90045e895d9e6bc8336f38d": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HTMLModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HTMLView",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_a1f9891798f948f783e38da6c496ba72",
"placeholder": "​",
"style": "IPY_MODEL_3c7d426404d94674b0dbdac23fe63877",
"value": " 90.9M/90.9M [00:01&lt;00:00, 94.0MB/s]"
}
},
"3b17bfc82268496db611d51f12596f66": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ProgressStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "ProgressStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"bar_color": null,
"description_width": ""
}
},
"3b5135d323ef445a87c881f324e16775": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ProgressStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "ProgressStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"bar_color": null,
"description_width": ""
}
},
"3c2b352b0c9a4e8992caf98270fdf3e5": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HTMLModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HTMLView",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_19fe2169977f42d48fa87bba7c24f093",
"placeholder": "​",
"style": "IPY_MODEL_f188cb08e688459da3b25e1f6a91e773",
"value": " 350/350 [00:00&lt;00:00, 11.1kB/s]"
}
},
"3c7d426404d94674b0dbdac23fe63877": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "DescriptionStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"description_width": ""
}
},
"3e643a1c7d024db5a2812e099ff31d5d": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "DescriptionStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"description_width": ""
}
},
"3ff36bea6003489b9d305a1c72c53e3b": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HTMLModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HTMLView",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_a2ee82dc8889414cb8ad80648d95ea7f",
"placeholder": "​",
"style": "IPY_MODEL_fbbaccf1af6c4843b419ea826bfbbe33",
"value": "Downloading (…)okenizer_config.json: 100%"
}
},
"443f5ae986364eadb7f97554004ad905": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "DescriptionStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"description_width": ""
}
},
"4e301f7b20ba460e9ac04e404aedbfd1": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"5930e9ce020b44688e8f24eb67cc3c98": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"5e50f788bb2041f1a6c80a8ed9979859": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ProgressStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "ProgressStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"bar_color": null,
"description_width": ""
}
},
"623da5cb6acb443ca18df8aca5b76ea8": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "FloatProgressModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "FloatProgressModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "ProgressView",
"bar_style": "success",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_d4f2e3100b30488c9eec6e7fe8353214",
"max": 231508,
"min": 0,
"orientation": "horizontal",
"style": "IPY_MODEL_17ebec0847cd45068d51aea8b91b942a",
"value": 231508
}
},
"670deb0ee14e4d358f875284307db5cd": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "DescriptionStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"description_width": ""
}
},
"67af2c52fb604dbe98d8ea3293a7df20": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "DescriptionStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"description_width": ""
}
},
"68ccf6327d124f7784c35a8c6457fda9": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HTMLModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HTMLView",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_902bcfeda02149e780fa2c331c568fa9",
"placeholder": "​",
"style": "IPY_MODEL_32635200eb334b33854bfb92a887a0e7",
"value": " 112/112 [00:00&lt;00:00, 4.09kB/s]"
}
},
"6e4ae30d23df4c428761869351ec8aba": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HBoxModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HBoxModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HBoxView",
"box_style": "",
"children": [
"IPY_MODEL_70ce835b56c94c3cb8a00702bc10c5b3",
"IPY_MODEL_29c5530a3c6a4f2981eec6c83e1ed3b8",
"IPY_MODEL_68ccf6327d124f7784c35a8c6457fda9"
],
"layout": "IPY_MODEL_4e301f7b20ba460e9ac04e404aedbfd1"
}
},
"6fb958559eb24d88a60c8a58d8e0b4fd": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HTMLModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HTMLView",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_02e7dd085a5548fe9276dacd6dac44d3",
"placeholder": "​",
"style": "IPY_MODEL_d35730a2d0b44554b25895e583002a44",
"value": " 232k/232k [00:00&lt;00:00, 3.02MB/s]"
}
},
"70ce835b56c94c3cb8a00702bc10c5b3": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HTMLModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HTMLView",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_c0f89fda1ae746f181d441c7efbf396b",
"placeholder": "​",
"style": "IPY_MODEL_670deb0ee14e4d358f875284307db5cd",
"value": "Downloading (…)cial_tokens_map.json: 100%"
}
},
"72f2f71bcd72468689d844da06a446cf": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"74a8031f0d074ebcbde72b412ae1ac75": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HTMLModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HTMLView",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_5930e9ce020b44688e8f24eb67cc3c98",
"placeholder": "​",
"style": "IPY_MODEL_eed379e58e2b4040b23ae3a466b46cdd",
"value": "Downloading (…)solve/main/vocab.txt: 100%"
}
},
"7cc326c04d7e4a7883254d8b3181246b": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HTMLModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HTMLView",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_95e2f366007c42c68a826a031025bdd6",
"placeholder": "​",
"style": "IPY_MODEL_67af2c52fb604dbe98d8ea3293a7df20",
"value": "Downloading pytorch_model.bin: 100%"
}
},
"7d5a3351f8904eb9a93d4f52db886479": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "FloatProgressModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "FloatProgressModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "ProgressView",
"bar_style": "success",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_b1bd0af5e7a64ba1860bd34374b04e86",
"max": 612,
"min": 0,
"orientation": "horizontal",
"style": "IPY_MODEL_3b5135d323ef445a87c881f324e16775",
"value": 612
}
},
"8b27c489dfcb4177a810f1882d141512": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ProgressStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "ProgressStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"bar_color": null,
"description_width": ""
}
},
"902bcfeda02149e780fa2c331c568fa9": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"95e2f366007c42c68a826a031025bdd6": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"97372744e58a4248baa032a0124a609b": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"9ac361cf979642c796b078db59c3c16f": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"9b3fc063fc654031b69d46f662e4c276": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HBoxModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HBoxModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HBoxView",
"box_style": "",
"children": [
"IPY_MODEL_3ff36bea6003489b9d305a1c72c53e3b",
"IPY_MODEL_d3a5b4f5c55e4d508a8d5a86cc285baf",
"IPY_MODEL_3c2b352b0c9a4e8992caf98270fdf3e5"
],
"layout": "IPY_MODEL_e8cba6eaa40f4ff9ab3bfd58e791d4d5"
}
},
"a1f9891798f948f783e38da6c496ba72": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"a2ee82dc8889414cb8ad80648d95ea7f": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"a37734b7f543488bb9441cd186226948": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"aed8eedffd34480b81075da97630ae9c": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HTMLModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HTMLView",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_ee863a159b8b4bfb8fac96fa54d19182",
"placeholder": "​",
"style": "IPY_MODEL_31569166f8614a4b98fa21095e7a085f",
"value": " 466k/466k [00:00&lt;00:00, 4.65MB/s]"
}
},
"b1bd0af5e7a64ba1860bd34374b04e86": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"b1dde0f6753f4565b261edc3cc37d130": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HBoxModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HBoxModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HBoxView",
"box_style": "",
"children": [
"IPY_MODEL_f4a320d5ea554a8788e8aad9289d810e",
"IPY_MODEL_7d5a3351f8904eb9a93d4f52db886479",
"IPY_MODEL_fc34571ed15e44d2909b252370d44609"
],
"layout": "IPY_MODEL_230429d7ebfe4b659c23ba1b7f292cc0"
}
},
"b3c5a224cef04ecd93b0f892e70dfc37": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "FloatProgressModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "FloatProgressModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "ProgressView",
"bar_style": "success",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_cce2022a43fd4405ba7b8de02382753e",
"max": 90888945,
"min": 0,
"orientation": "horizontal",
"style": "IPY_MODEL_5e50f788bb2041f1a6c80a8ed9979859",
"value": 90888945
}
},
"ba0d451eb6764795b95dfbf14aff5b0d": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "FloatProgressModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "FloatProgressModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "ProgressView",
"bar_style": "success",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_eb8d6819a0d2419d9cfc72f146cda2ea",
"max": 466247,
"min": 0,
"orientation": "horizontal",
"style": "IPY_MODEL_8b27c489dfcb4177a810f1882d141512",
"value": 466247
}
},
"bed72188190c49598606622abd9ea923": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"c0f89fda1ae746f181d441c7efbf396b": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"c1554fc9092b4520982978171adf2fb2": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "DescriptionStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"description_width": ""
}
},
"c586ec7187314dbc9d5cc4c90f703135": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"c5fc977b656945369d95d59258fe8eb8": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ProgressStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "ProgressStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"bar_color": null,
"description_width": ""
}
},
"c702c4db373b4eb0a5f131ee58ccf5c3": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HBoxModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HBoxModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HBoxView",
"box_style": "",
"children": [
"IPY_MODEL_74a8031f0d074ebcbde72b412ae1ac75",
"IPY_MODEL_623da5cb6acb443ca18df8aca5b76ea8",
"IPY_MODEL_6fb958559eb24d88a60c8a58d8e0b4fd"
],
"layout": "IPY_MODEL_bed72188190c49598606622abd9ea923"
}
},
"cce2022a43fd4405ba7b8de02382753e": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"d35730a2d0b44554b25895e583002a44": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "DescriptionStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"description_width": ""
}
},
"d3a5b4f5c55e4d508a8d5a86cc285baf": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "FloatProgressModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "FloatProgressModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "ProgressView",
"bar_style": "success",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_c586ec7187314dbc9d5cc4c90f703135",
"max": 350,
"min": 0,
"orientation": "horizontal",
"style": "IPY_MODEL_c5fc977b656945369d95d59258fe8eb8",
"value": 350
}
},
"d3bb5e665cb04e25b35e250df0a025f0": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"d4f2e3100b30488c9eec6e7fe8353214": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"d601874457a84180b28c40d4defdbef8": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HBoxModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HBoxModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HBoxView",
"box_style": "",
"children": [
"IPY_MODEL_7cc326c04d7e4a7883254d8b3181246b",
"IPY_MODEL_b3c5a224cef04ecd93b0f892e70dfc37",
"IPY_MODEL_37e04779c90045e895d9e6bc8336f38d"
],
"layout": "IPY_MODEL_a37734b7f543488bb9441cd186226948"
}
},
"e8cba6eaa40f4ff9ab3bfd58e791d4d5": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"eb8d6819a0d2419d9cfc72f146cda2ea": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"ee863a159b8b4bfb8fac96fa54d19182": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"eed379e58e2b4040b23ae3a466b46cdd": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "DescriptionStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"description_width": ""
}
},
"f188cb08e688459da3b25e1f6a91e773": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "DescriptionStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"description_width": ""
}
},
"f4a320d5ea554a8788e8aad9289d810e": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HTMLModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HTMLView",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_97372744e58a4248baa032a0124a609b",
"placeholder": "​",
"style": "IPY_MODEL_c1554fc9092b4520982978171adf2fb2",
"value": "Downloading (…)lve/main/config.json: 100%"
}
},
"f5e47bfbaef0488bbb0f1ef239facd32": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HTMLModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HTMLView",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_9ac361cf979642c796b078db59c3c16f",
"placeholder": "​",
"style": "IPY_MODEL_443f5ae986364eadb7f97554004ad905",
"value": "Downloading (…)/main/tokenizer.json: 100%"
}
},
"fbbaccf1af6c4843b419ea826bfbbe33": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "DescriptionStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"description_width": ""
}
},
"fc34571ed15e44d2909b252370d44609": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HTMLModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HTMLView",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_1b03824f9d814f799679cb9669b3f276",
"placeholder": "​",
"style": "IPY_MODEL_3e643a1c7d024db5a2812e099ff31d5d",
"value": " 612/612 [00:00&lt;00:00, 12.6kB/s]"
}
}
}
}
},
"nbformat": 4,
"nbformat_minor": 0
}
FrontEnd/lib/screens/register.dart 0 → 100644
View file @ 5d53e813
import 'package:flutter/gestures.dart';
import 'package:flutter/material.dart';
import 'package:govimithura/models/error_model.dart';
import 'package:govimithura/screens/login.dart';
import 'package:govimithura/utils/loading_overlay.dart';
import 'package:govimithura/utils/utils.dart';
import 'package:provider/provider.dart';
import '../providers/authentication_provider.dart';
import '../utils/screen_size.dart';
import '../widgets/utils/buttons/custom_elevated_button.dart';
import '../widgets/utils/common_widget.dart';
import '../widgets/utils/text_fields/primary_textfield.dart';
class RegisterScreen extends StatefulWidget {
const RegisterScreen({super.key});
@override
State<RegisterScreen> createState() => _RegisterScreenState();
}
class _RegisterScreenState extends State<RegisterScreen> {
late AuthenticationProvider pAuthentication;
GlobalKey<FormState> registerFormKey = GlobalKey<FormState>();
@override
void initState() {
super.initState();
pAuthentication =
Provider.of<AuthenticationProvider>(context, listen: false);
}
@override
Widget build(BuildContext context) {
return Scaffold(
body: SingleChildScrollView(
child: SizedBox(
height: ScreenSize.height,
width: ScreenSize.width,
child: Form(
key: registerFormKey,
child: Column(
mainAxisSize: MainAxisSize.max,
mainAxisAlignment: MainAxisAlignment.center,
children: [
Padding(
padding: const EdgeInsets.symmetric(horizontal: 20),
child: Column(
children: [
const LogoText(),
spacingWidget(30, SpaceDirection.vertical),
Text(
'Welcome to Govimithura',
style: Theme.of(context).textTheme.titleLarge,
),
spacingWidget(20, SpaceDirection.vertical),
PrimaryTextField(
hintText: "First name",
validator: (value) {
if (value == null || value.isEmpty) {
return 'Required';
}
return null;
},
onChanged: (value) {
pAuthentication.setFirstName(value.trim());
},
),
spacingWidget(20, SpaceDirection.vertical),
PrimaryTextField(
hintText: "Last name",
validator: (value) {
if (value == null || value.isEmpty) {
return 'Required';
}
return null;
},
onChanged: (value) {
pAuthentication.setLastName(value.trim());
},
),
spacingWidget(20, SpaceDirection.vertical),
PrimaryTextField(
hintText: "Email address",
validator: (value) {
if (value == null || value.isEmpty) {
return 'Required';
} else if (!(value.contains("@") &&
value.contains("."))) {
return 'Enter valid Email Address';
}
return null;
},
onChanged: (value) {
pAuthentication.setEmail(value.trim());
},
),
spacingWidget(20, SpaceDirection.vertical),
PrimaryTextField(
isPassword: true,
hintText: "Password",
validator: (value) {
if (value == null || value.isEmpty) {
return 'Required';
}
return null;
},
onChanged: (value) {
pAuthentication.setPassWord(value.trim());
},
),
spacingWidget(20, SpaceDirection.vertical),
PrimaryTextField(
isPassword: true,
hintText: "Confirm password",
validator: (value) {
if (value == null || value.isEmpty) {
return 'Required';
} else if (value !=
pAuthentication.authModel.password) {
return "Password doesn't match";
}
return null;
},
onChanged: (value) {
pAuthentication.setConfirmPassword(value.trim());
},
),
spacingWidget(30, SpaceDirection.vertical),
CustomElevatedButton(
text: "Register ",
onPressed: () async {
if (registerFormKey.currentState!.validate()) {
LoadingOverlay overlay = LoadingOverlay.of(context);
bool success = await overlay
.during(pAuthentication.register());
if (success) {
if (context.mounted) {
Navigator.pushReplacement(
context,
MaterialPageRoute(
builder: (context) => const LoginScreen(),
),
);
}
} else {
if (context.mounted) {
Utils.showSnackBar(
context, ErrorModel.errorMessage);
}
}
}
},
),
spacingWidget(20, SpaceDirection.vertical),
RichText(
text: TextSpan(
text: "Already have an account? ",
style: Theme.of(context).textTheme.bodyLarge,
children: [
TextSpan(
text: "Login",
style: Theme.of(context)
.textTheme
.bodyLarge
?.copyWith(
color: Theme.of(context).primaryColor,
),
recognizer: TapGestureRecognizer()
..onTap = () {
Navigator.pushReplacement(
context,
MaterialPageRoute(
builder: (context) => const LoginScreen(),
),
);
},
),
],
),
)
],
),
),
],
),
),
),
),
);
}
}
README.md
View file @ 5d53e813
Main Objective:
...Research problem...
The Main Objective of the research topic is even a person with primary knowledge
or without any knowledge about home gardening can use this mobile application to make home
gardening in effective way to reap the harvest.
Agriculture is one of the major livelihoods in Sri Lanka. People are engaging in cultivation in commercial gardens and also at a smaller level, in home gardens.
Due to the current economic and fuel crisis in Sri Lanka, people have found it difficult to get their daily food needs from stores. because of these reasons again people have turned their attention on home gardening.
Research Problem:
Sri Lankan people have found it difficult to get their home gardening inquiry solutions in their day-to-day life. There is no such application developed for Sri Lankan home gardening to help users. The main research problem is to answer the necessary home gardening inquiries through message commands in the chatbot.
Due to the current economic and fuel crisis, most of the people living in Sri Lanka are now turning to home gardening.
But the main problem here is that people who are used to a modern lifestyle do not have proper knowledge about gardening.
There is no path to take decisions and advice about problems that arise when they start home gardening in their own living places.
Due to these facts, people cannot reap a bountiful harvest from the garden.
...Main Objective...
If we consider these points in more detail, the problem is that many people who turn to home gardening today, have a rudimentary knowledge
of home gardening and lack the proper understanding of home gardening and climate changes and crop growing quarters. Therefore, People who
cultivate various crops have not been able to harvest as much as they expected.
The main objective of this home gardening chatbot is to provide users in the agriculture industry with quick and reliable information on various topics such as crop management, weather conditions, and watering plants, among others. The aim is to improve productivity and sustainability in home gardening.
Another problem faced by people is that people are not able to analyze the disease affecting a particular crop. Once the disease affects
the crop, the growth of some stems of the crop is reduced, but the disease can still attack the crop. So those whole crops or parts are
destroyed and much of their produce is reduced before harvest. Therefore, people don’t have little understanding of what remedies are needed to
prevent the spread of the disease.
...Sub Objectives...
And also, another problem that arises in day-to-day home gardening activities, people face various problems in crop cultivation mainly due to
their rudimentary knowledge about gardening and unable to identify the disease affecting a certain crop, people face various problems in crop
cultivation and also people use different types of fertilizers, herbicides, and insecticides while growing crops. They don’t know how to use
them properly for the crops. Also, due to the fuel crisis in the country, people have not been able to go to the agricultural service centers
to get necessary advice related to home gardening and above mention problems.
1. Provide small details of the home gardening inquiries and improving users' knowledge and skills through educational content.
2. Natural Language Processing model to answer inquires.
3. Develop a pre trained model to identify keywords in the given questions.
4. Then extracted documents from the Wikipedia for the identified keywords.
5. Develop a mobile application and integrate with the developed models.
The other major problem affecting home gardening is considered to be the destruction of crops by animals, so the severity problem is attacks
of wild animals. For example, elephants, peacocks, monkeys, greater coucal, and so on. They cause crop damage. Due to that reason, people will not be
able to expect a harvest from gardening. Due to the lack of a proper remedy for it, the people who are currently engaged in home gardening activities
are hindered from cultivating successfully.
...User Requirements...
Individual research question:
1. How do people cultivate crops harvested as they accepted, according to crop analysis and prediction?
2. How do people recognize the affected crop diseases and get relevant solutions using computer vision?
3. How people find the right solutions to their queries through different languages via chat messaging and voice messaging?
4. How to identify and repel harmful animals that come to the plantation?
1. The user should be able to get accurate and up-to-date information related to agriculture such as crop information, weather conditions, and pest control.
2. The chatbot should be easily accessible and convenient to use.
3. The chatbot should have a user-friendly interface that is easy to navigate and understand.
Individual Objectives:
1. Crop analysis and predictions.
People may have a lack of proper understanding of home gardening and climate changes and crop growing quarters.
Therefore, People who cultivate various crops have not been able to harvest as much as they expected.
Due to that problem the user will be creating their own account while adding the location where the person lives
and the relevant quarter of their own. The location of the user will be stored in the system. After that, according
to the user's above information of predicting the type of crop suitable for cultivation at that time and giving instructions related to cultivation.
2. Crop diseases recognition based on Computer vision.
Select the leaf affected by disease and then collect the leaf from the plant and take a photo of the leaf and upload the leaf image to the system.
Preprocessing the input image to improve image quality and remove unwanted distortions from the image. Through the approach of plant disease identification
using image processing and machine learning is illustrated in Image preprocessing, segmentation, feature extraction, and classification.
And also, we can include several classification algorithms like NB, KNN, DT, SVM, and RF to identify the diagnosis of disease and give solutions and recommendations.
3. Multipurpose Chat Application.
In the study of the research papers on this Chat Applications, it was the latest digital interface developments, for the growth of the web and smartphone apps.
It is well documented for these applications to use automatic conversational agents that operate on software creation or a kind of artificial intelligence (AI)
relationship between users and automated systems. This AI-based multi-purpose chat application helps users to find the right answers to their queries to make the
right decisions. This multi-purpose chat application is made by providing feedback in two different languages and voice activation.
And also users can multilingual both text and voice programmable in this chat application. This system will enable the users to ask any number of inquiries,
which will help to spread the latest Home Gardening technology faster and to a higher number of users.
4. Animal detective camera security system.
Gardening is constantly monitored by a security camera system. This will constantly monitor the inner as well as the outer boundary of the gardening.
Due to this process, if an animal that damages the plantation reaches the outer limit of the plantation or enters the plantation,
this security camera system will identify the animal that has entered the plantation and inform the user of the location of the animal.
Because of this, the gardener does not need to keep an eye on his crops constantly to protect them from harmful animals.
Here, by installing speakers at selected locations around the gardening, the security camera system can identify the relevant animal and
emit sounds necessary to drive away the relevant animal.
(Ex: making a blast sound to drive away animals like peacocks, monkeys, and pigs and making the sound of bees to drive away wild elephants)
This measure will be able to reduce the damage caused to crops, especially by animals like elephants, peacocks, monkeys, cows
...Functional Requirements...
1. The chatbot should be able to provide personalized information to users based on their specific requirements in the questions.
2. The chatbot should use NLP to understand and respond to user queries in a conversational manner.
3. The chatbot should be able to retrieve and access relevant information from the internet related to agriculture.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment