Commit af75b08f authored by Wickramasinghe R.J.P's avatar Wickramasinghe R.J.P

NLP module added

parent 584861af
# Default ignored files
/shelf/
/workspace.xml
# Editor-based HTTP Client requests
/httpRequests/
# Datasource local storage ignored files
/dataSources/
/dataSources.local.xml
<?xml version="1.0" encoding="UTF-8"?>
<module type="PYTHON_MODULE" version="4">
<component name="NewModuleRootManager">
<content url="file://$MODULE_DIR$">
<excludeFolder url="file://$MODULE_DIR$/venv" />
</content>
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
</component>
<component name="PyDocumentationSettings">
<option name="renderExternalDocumentation" value="true" />
</component>
</module>
\ No newline at end of file
<component name="InspectionProjectProfileManager">
<settings>
<option name="USE_PROJECT_PROFILE" value="false" />
<version value="1.0" />
</settings>
</component>
\ No newline at end of file
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ProjectRootManager" version="2" project-jdk-name="Python 3.10 (NLPTechniques)" project-jdk-type="Python SDK" />
</project>
\ No newline at end of file
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ProjectModuleManager">
<modules>
<module fileurl="file://$PROJECT_DIR$/.idea/NLPTechniques.iml" filepath="$PROJECT_DIR$/.idea/NLPTechniques.iml" />
</modules>
</component>
</project>
\ No newline at end of file
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="PySciProjectComponent">
<option name="PY_SCI_VIEW" value="true" />
<option name="PY_SCI_VIEW_SUGGESTED" value="true" />
</component>
</project>
\ No newline at end of file
Name,Info
Dog Skin Problems ,The sound of a dog constantly scratching or licking can be as irritating as nails on a chalkboard. But don’t blame your pooch for these bad habits -- a skin condition is probably the culprit. Possible causes range from parasites to allergies to underlying illness. WebMD has compiled images of some of the most common canine skin problems.
Allergic Dermatitis ,"Dogs can have allergic reactions to grooming products, food, and environmental irritants, such as pollen or insect bites. A dog with allergies may scratch relentlessly, and a peek at the skin often reveals an ugly rash. Corticosteroids or other, newer medicines can help with itchy rashes. But the most effective treatment is to identify and avoid exposure to the allergens."
Yeast Infection ,"If your dog can't seem to stop scratching an ear or licking and chewing their toes, ask your veterinarian to check for a yeast infection. Symptoms include irritated, itchy, or discolored skin. The infection usually strikes the paws or ears, where yeast have a cozy space to grow. Yeast infections are easy to diagnose and often respond well to a topical cream. In some cases, your veterinarian may prescribe oral drugs, medicated sprays, or medicated baths."
Folliculitis ,"Superficial bacterial folliculitis is an infection that causes sores, bumps, and scabs on the skin. These skin abnormalities are easier to see in shorthaired dogs. In longhaired dogs, the most obvious symptoms may be a dull coat and shedding with scaly skin underneath. Folliculitis often occurs in conjunction with other skin problems, such as mange, allergies, or injury. Treatment may include oral antibiotics and antibacterial ointments or shampoos."
Impetigo ,"Another type of bacterial infection, impetigo is most common in puppies. It causes pus-filled blisters that may break and crust over. The blisters usually develop on the hairless portion of the abdomen. Impetigo is rarely serious and can be treated with a topical solution. In a small number of cases, the infection may spread or persist."
Seborrhea ,"Seborrhea causes a dog's skin to become greasy and develop scales (dandruff). In some cases, it's a genetic disease that begins when a dog is young and lasts a lifetime. But most dogs with seborrhea develop the scaling as a complication of another medical problem, such as allergies or hormonal abnormalities. In these cases, it is vital to treat the underlying cause so symptoms do not recur. The seborrhea itself typically can be treated with certain medicated shampoos."
Ringworm ,"Despite its name, ringworm is not caused by a worm, but by a fungus. The term ""ring"" comes from the circular patches that can form anywhere, but are often found on a dog's head, paws, ears, and forelegs. Inflammation, scaly patches, and hair loss often surround the lesions. Puppies less than a year old are the most susceptible, and the infection can spread quickly between dogs in a kennel or to pet owners at home. Various anti-fungal treatments are available."
Shedding and Hair Loss (Alopecia) ,"Anyone who shares their home with dogs knows that they shed. How much shedding is normal depends on breed, time of year, and environment. But sometimes stress, poor nutrition, or illness can cause a dog to lose more hair than usual. If abnormal or excessive shedding persists for more than a week, or you notice patches of missing fur, check with your veterinarian."
Mange (Mites) ,Mange is a skin disorder caused by tiny parasites called mites.
Fleas ,"Fleas are the bane of any pet owner. You may not see the tiny insects themselves, but flea droppings or eggs are usually visible in a dog's coat. Other symptoms include excessive licking or scratching, scabs, and hot spots. Severe flea infestations can cause blood loss and anemia, and even expose your dog to other parasites, such as tapeworms. Treatment may include a topical and/or oral flea killer and a thorough cleaning of the pet's home and yard."
Ticks ,"Ticks, like fleas, are external parasites that feed on the blood of their hosts. You can spot a tick feeding on your dog with the naked eye. To properly remove a tick, grasp the tick with tweezers close to the dog’s skin, and gently pull it straight out. Twisting or pulling too hard may cause the head to remain lodged in your dog’s skin, which can lead to infection. Place the tick in a jar with some alcohol for a couple of days. If your pet gets ill, your vet may need it to analyze what's wrong. In addition to causing blood loss and anemia, ticks can transmit Lyme disease and other potentially serious bacterial infections. If you live in an area where ticks are common, talk to your veterinarian about tick control products."
Color or Texture Changes ,Changes in a dog's skin color or coat texture can be a warning sign of several common metabolic or hormone problems. They can also result from an infection or other skin disorder. Usually a simple blood test can identify the underlying cause. Be sure to ask your veterinarian about any significant changes to your dog’s coat.
" Dry, Flaky Skin ","Dry, flaky skin can be a red flag for a number of problems. It's a common symptom of allergies, mange, and other skin diseases. But most often, dry or flaky skin is nothing serious. Make sure you are feeding Fido high quality food. Like people, some dogs simply get dry skin in the winter. If this seems to cause your pet discomfort, consult your veterinarian. Ask whether a fatty acid supplement or a humidifier might help."
Acral Lick Granuloma ,"Also called acral lick dermatitis, this is a frustrating skin condition caused by compulsive, relentless licking of a single area -- most often on the front of the lower leg. The area is unable to heal, and the resulting pain and itching can lead the dog to keep licking the same spot. Treatment includes discouraging the dog from licking, either by using a bad-tasting topical solution or an Elizabethan collar. Also ask your dog's vet about other treatment options."
Skin Tumors ,"If you notice a lump on your dog's skin, point it out to your vet as soon as possible. Dogs can develop cancerous tumors in their skin. The only way to confirm a diagnosis of cancer is to biopsy the tumor. If the lump is small enough, your veterinarian may recommend removing it entirely. This can yield a diagnosis and treatment with a single procedure. For tumors that have not spread, this may be the only treatment needed."
Hot Spots ,"Hot spots, also called acute moist dermatitis, are small areas that appear red, irritated, and inflamed. They are most commonly found on a dog's head, hips, or chest, and often feel hot to the touch. Hot spots can result from a wide range of conditions, including infections, allergies, insect bites, or excessive licking and chewing. Treatment consists of cleansing the hot spot and addressing the underlying condition."
Immune Disorders ,"In rare cases, skin lesions or infections that won’t heal can indicate an immune disorder in your dog. One of the best known is lupus, a disease that affects dogs and people. Lupus is an autoimmune disorder, meaning the body’s immune system attacks its own cells. Symptoms include skin abnormalities and kidney problems. It can be fatal if untreated."
Anal Sac Disease ,"As if dog poop weren't smelly enough, dogs release a foul-smelling substance when they do their business. The substance comes from small anal sacs, which can become impacted if they don't empty properly. The hallmark of impacted anal sacs is a dog scooting their bottom along the ground. Other symptoms include biting or licking the anal area. A vet can manually express full anal sacs, but in severe cases, the sacs may be surgically removed."
When to See the Vet ,"Although most skin problems are not emergencies, it is important to get an accurate diagnosis so the condition can be treated. See your veterinarian if your dog is scratching or licking excessively, or if you notice any changes in your pet's coat or skin, including scaling, redness, discoloration, or bald patches. Once the cause is identified, most skin problems respond well to treatment."
,Name,lemmatized
1, Allergic Dermatitis ,"['dog', 'allergic', 'reaction', 'groom', 'product', 'food', 'environmental', 'irritant', 'pollen', 'insect', 'bite', 'dog', 'allergy', 'may', 'scratch', 'relentlessly', 'peek', 'skin', 'often', 'reveal', 'ugly', 'rash', 'corticosteroid', 'new', 'medicine', 'help', 'itchy', 'rashes.â', 'effective', 'treatment', 'identify', 'avoid', 'exposure', 'allergen']"
2, Yeast Infection ,"['dog', 'seem', 'stop', 'scratch', 'ear', 'lick', 'chewing', 'toe', 'ask', 'veterinarian', 'check', 'yeast', 'infection', 'symptom', 'include', 'irritated', 'itchy', 'discolor', 'skin', 'infection', 'usually', 'strike', 'paws', 'ear', 'yeast', 'cozy', 'space', 'grow', 'yeast', 'infection', 'easy', 'diagnose', 'often', 'respond', 'well', 'topical', 'cream', 'case', 'veterinarian', 'may', 'prescribe', 'oral', 'drug', 'medicate', 'spray', 'â', 'medicate', 'bath']"
3, Folliculitis ,"['superficial', 'bacterial', 'folliculitis', 'infection', 'cause', 'sore', 'bump', 'scab', 'skin', 'skin', 'abnormality', 'easier', 'see', 'shorthaired', 'dog', 'longhaired', 'dog', 'obvious', 'symptom', 'may', 'dull', 'coat', 'shed', 'scaly', 'skin', 'underneath', 'folliculitis', 'often', 'occur', 'conjunction', 'skin', 'problem', 'mange', 'allergy', 'injury', 'treatment', 'may', 'include', 'oral', 'antibiotic', 'antibacterial', 'ointment', 'shampoo']"
4, Impetigo ,"['another', 'type', 'bacterial', 'infection', 'impetigo', 'common', 'puppy', 'cause', 'pus-filled', 'blister', 'may', 'break', 'crust', 'blister', 'usually', 'develop', 'hairless', 'portion', 'abdomen', 'impetigo', 'rarely', 'serious', 'treat', 'topical', 'solution', 'small', 'number', 'case', 'infection', 'may', 'spread', 'persist']"
5, Seborrhea ,"['seborrhea', 'cause', 'dog', ""'s"", 'skin', 'become', 'greasy', 'develop', 'scale', 'dandruff', 'case', 'genetic', 'disease', 'begin', 'dog', 'young', 'last', 'lifetime', 'dog', 'seborrhea', 'develop', 'scale', 'complication', 'another', 'medical', 'problem', 'allergy', 'hormonal', 'abnormality', 'case', 'vital', 'treat', 'underlie', 'symptom', 'recur', 'seborrhea', 'typically', 'treat', 'certain', 'medicated', 'shampoo']"
6, Ringworm ,"['despite', 'name', 'ringworm', 'cause', 'worm', 'fungus', 'term', '``', 'ring', ""''"", 'come', 'circular', 'patch', 'form', 'anywhere', 'often', 'find', 'dog', ""'s"", 'head', 'paw', 'ears', 'forelegs', 'inflammation', 'scaly', 'patch', 'hair', 'loss', 'often', 'surround', 'lesion', 'puppy', 'less', 'year', 'old', 'susceptible', 'infection', 'spread', 'quickly', 'dog', 'kennel', 'pet', 'owner', 'home', 'various', 'anti-fungal', 'treatment', 'available']"
7, Shedding and Hair Loss (Alopecia) ,"['anyone', 'share', 'home', 'dog', 'know', 'would', 'much', 'shed', 'normal', 'depends', 'breed', 'time', 'year', 'environment', 'sometimes', 'stress', 'poor', 'nutrition', 'illness', 'dog', 'lose', 'hair', 'usual', 'abnormal', 'excessive', 'shed', 'persists', 'week', 'notice', 'patch', 'miss', 'fur', 'check', 'veterinarian']"
8, Mange (Mites) ,"['mange', 'skin', 'disorder', 'cause', 'tiny', 'parasite', 'call', 'mite']"
9, Fleas ,"['flea', 'bane', 'pet', 'owner', 'may', 'see', 'tiny', 'insect', 'flea', 'droppings', 'egg', 'usually', 'visible', 'dog', ""'s"", 'coat', 'symptom', 'include', 'excessive', 'lick', 'scratch', 'scabs', 'hot', 'spot', 'severe', 'flea', 'infestation', 'blood', 'loss', 'anemia', 'even', 'expose', 'dog', 'parasite', 'tapeworm', 'treatment', 'may', 'include', 'topical', 'and/or', 'oral', 'flea', 'killer', 'thorough', 'cleaning', 'pet', ""'s"", 'home', 'yard']"
10, Ticks ,"['tick', 'like', 'fleas', 'external', 'parasite', 'fee', 'blood', 'host', 'spot', 'tick', 'feeding', 'dog', 'naked', 'eye', 'properly', 'remove', 'tick', 'grasp', 'tick', 'tweezer', 'close', 'dog’s', 'skin', 'gently', 'pull', 'straight', 'twist', 'pull', 'hard', 'may', 'head', 'remain', 'lodged', 'dog’s', 'skin', 'lead', 'infection', 'place', 'tick', 'jar', 'alcohol', 'couple', 'day', 'pet', 'get', 'ill', 'vet', 'may', 'need', 'analyze', 'wrong.â', 'addition', 'cause', 'blood', 'loss', 'anemia', 'tick', 'transmit', 'lyme', 'disease', 'potentially', 'serious', 'bacterial', 'infection', 'live', 'area', 'tick', 'common', 'talk', 'veterinarian', 'tick', 'control', 'product']"
11, Color or Texture Changes ,"['change', 'dog', ""'s"", 'skin', 'color', 'coat', 'texture', 'warn', 'sign', 'several', 'common', 'metabolic', 'hormone', 'problem', 'also', 'result', 'infection', 'skin', 'disorder', 'usually', 'simple', 'blood', 'test', 'identify', 'underlie', 'because.â', 'sure', 'ask', 'veterinarian', 'significant', 'change', 'dog’s', 'coat']"
12," Dry, Flaky Skin ","['dry', 'flaky', 'skin', 'red', 'flag', 'number', 'problem', 'common', 'symptom', 'allergy', 'mange', 'skin', 'disease', 'often', 'dry', 'flaky', 'skin', 'nothing', 'serious', 'make', 'sure', 'feed', 'fido', 'high', 'quality', 'food', 'like', 'people', 'dog', 'simply', 'get', 'dry', 'skin', 'winter', 'seem', 'pet', 'discomfort', 'consult', 'veterinarian', 'ask', 'whether', 'fatty', 'acid', 'supplementâ', 'humidifier', 'might', 'help']"
13, Acral Lick Granuloma ,"['also', 'call', 'acral', 'lick', 'dermatitis', 'frustrate', 'skin', 'condition', 'cause', 'compulsive', 'relentless', 'lick', 'single', 'area', '--', 'often', 'front', 'low', 'leg', 'area', 'unable', 'heal', 'result', 'pain', 'itch', 'lead', 'dog', 'keep', 'lick', 'spot', 'treatment', 'include', 'discourage', 'dog', 'lick', 'either', 'use', 'bad-tasting', 'topical', 'solution', 'elizabethan', 'collar', 'also', 'ask', 'dog', ""'s"", 'vet', 'treatment', 'option']"
14, Skin Tumors ,"['notice', 'lump', 'dog', ""'s"", 'skin', 'point', 'vet', 'soon', 'possible', 'dog', 'develop', 'cancerous', 'tumor', 'skin', 'way', 'confirm', 'diagnosis', 'cancer', 'biopsy', 'tumor', 'lump', 'small', 'enough', 'veterinarian', 'may', 'recommend', 'remove', 'entirely', 'yield', 'diagnosis', 'treatment', 'single', 'procedure', 'tumor', 'spread', 'may', 'treatment', 'need']"
15, Hot Spots ,"['hot', 'spot', 'also', 'call', 'acute', 'moist', 'dermatitis', 'small', 'area', 'appear', 'red', 'irritated', 'inflame', 'commonly', 'find', 'dog', ""'s"", 'head', 'hips', 'chest', 'often', 'feel', 'hot', 'touch', 'hot', 'spot', 'result', 'wide', 'range', 'condition', 'include', 'infection', 'allergy', 'insect', 'bite', 'excessive', 'lick', 'chew', 'treatment', 'consist', 'cleanse', 'hot', 'spot', 'address', 'underlying', 'condition']"
,Name,lemmatized,keyword
0, Dog Skin Problems ,"['sound', 'dog', 'constantly', 'scratch', 'lick', 'irritate', 'nail', 'chalkboard', 'don’t', 'blame', 'pooch', 'bad', 'habit', '--', 'skin', 'condition', 'probably', 'culprit', 'possible', 'cause', 'range', 'parasite', 'allergy', 'underlie', 'illness', 'webmd', 'compile', 'image', 'common', 'canine', 'skin', 'problem']",
1, Allergic Dermatitis ,"['dog', 'allergic', 'reaction', 'groom', 'product', 'food', 'environmental', 'irritant', 'pollen', 'insect', 'bite', 'dog', 'allergy', 'may', 'scratch', 'relentlessly', 'peek', 'skin', 'often', 'reveal', 'ugly', 'rash', 'corticosteroid', 'new', 'medicine', 'help', 'itchy', 'rashes.â', 'effective', 'treatment', 'identify', 'avoid', 'exposure', 'allergen']","allergic, environmental"
2, Yeast Infection ,"['dog', 'seem', 'stop', 'scratch', 'ear', 'lick', 'chewing', 'toe', 'ask', 'veterinarian', 'check', 'yeast', 'infection', 'symptom', 'include', 'irritated', 'itchy', 'discolor', 'skin', 'infection', 'usually', 'strike', 'paws', 'ear', 'yeast', 'cozy', 'space', 'grow', 'yeast', 'infection', 'easy', 'diagnose', 'often', 'respond', 'well', 'topical', 'cream', 'case', 'veterinarian', 'may', 'prescribe', 'oral', 'drug', 'medicate', 'spray', 'â', 'medicate', 'bath']","infection, yeast"
3, Folliculitis ,"['superficial', 'bacterial', 'folliculitis', 'infection', 'cause', 'sore', 'bump', 'scab', 'skin', 'skin', 'abnormality', 'easier', 'see', 'shorthaired', 'dog', 'longhaired', 'dog', 'obvious', 'symptom', 'may', 'dull', 'coat', 'shed', 'scaly', 'skin', 'underneath', 'folliculitis', 'often', 'occur', 'conjunction', 'skin', 'problem', 'mange', 'allergy', 'injury', 'treatment', 'may', 'include', 'oral', 'antibiotic', 'antibacterial', 'ointment', 'shampoo']",infection
4, Impetigo ,"['another', 'type', 'bacterial', 'infection', 'impetigo', 'common', 'puppy', 'cause', 'pus-filled', 'blister', 'may', 'break', 'crust', 'blister', 'usually', 'develop', 'hairless', 'portion', 'abdomen', 'impetigo', 'rarely', 'serious', 'treat', 'topical', 'solution', 'small', 'number', 'case', 'infection', 'may', 'spread', 'persist']",infection
5, Seborrhea ,"['seborrhea', 'cause', 'dog', ""'s"", 'skin', 'become', 'greasy', 'develop', 'scale', 'dandruff', 'case', 'genetic', 'disease', 'begin', 'dog', 'young', 'last', 'lifetime', 'dog', 'seborrhea', 'develop', 'scale', 'complication', 'another', 'medical', 'problem', 'allergy', 'hormonal', 'abnormality', 'case', 'vital', 'treat', 'underlie', 'symptom', 'recur', 'seborrhea', 'typically', 'treat', 'certain', 'medicated', 'shampoo']",dandruff
6, Ringworm ,"['despite', 'name', 'ringworm', 'cause', 'worm', 'fungus', 'term', '``', 'ring', ""''"", 'come', 'circular', 'patch', 'form', 'anywhere', 'often', 'find', 'dog', ""'s"", 'head', 'paw', 'ears', 'forelegs', 'inflammation', 'scaly', 'patch', 'hair', 'loss', 'often', 'surround', 'lesion', 'puppy', 'less', 'year', 'old', 'susceptible', 'infection', 'spread', 'quickly', 'dog', 'kennel', 'pet', 'owner', 'home', 'various', 'anti-fungal', 'treatment', 'available']","fungus, infection"
7, Shedding and Hair Loss (Alopecia) ,"['anyone', 'share', 'home', 'dog', 'know', 'would', 'much', 'shed', 'normal', 'depends', 'breed', 'time', 'year', 'environment', 'sometimes', 'stress', 'poor', 'nutrition', 'illness', 'dog', 'lose', 'hair', 'usual', 'abnormal', 'excessive', 'shed', 'persists', 'week', 'notice', 'patch', 'miss', 'fur', 'check', 'veterinarian']",environment
8, Mange (Mites) ,"['mange', 'skin', 'disorder', 'cause', 'tiny', 'parasite', 'call', 'mite']",mite
9, Fleas ,"['flea', 'bane', 'pet', 'owner', 'may', 'see', 'tiny', 'insect', 'flea', 'droppings', 'egg', 'usually', 'visible', 'dog', ""'s"", 'coat', 'symptom', 'include', 'excessive', 'lick', 'scratch', 'scabs', 'hot', 'spot', 'severe', 'flea', 'infestation', 'blood', 'loss', 'anemia', 'even', 'expose', 'dog', 'parasite', 'tapeworm', 'treatment', 'may', 'include', 'topical', 'and/or', 'oral', 'flea', 'killer', 'thorough', 'cleaning', 'pet', ""'s"", 'home', 'yard']",flea
10, Ticks ,"['tick', 'like', 'fleas', 'external', 'parasite', 'fee', 'blood', 'host', 'spot', 'tick', 'feeding', 'dog', 'naked', 'eye', 'properly', 'remove', 'tick', 'grasp', 'tick', 'tweezer', 'close', 'dog’s', 'skin', 'gently', 'pull', 'straight', 'twist', 'pull', 'hard', 'may', 'head', 'remain', 'lodged', 'dog’s', 'skin', 'lead', 'infection', 'place', 'tick', 'jar', 'alcohol', 'couple', 'day', 'pet', 'get', 'ill', 'vet', 'may', 'need', 'analyze', 'wrong.â', 'addition', 'cause', 'blood', 'loss', 'anemia', 'tick', 'transmit', 'lyme', 'disease', 'potentially', 'serious', 'bacterial', 'infection', 'live', 'area', 'tick', 'common', 'talk', 'veterinarian', 'tick', 'control', 'product']","flea, infection"
11, Color or Texture Changes ,"['change', 'dog', ""'s"", 'skin', 'color', 'coat', 'texture', 'warn', 'sign', 'several', 'common', 'metabolic', 'hormone', 'problem', 'also', 'result', 'infection', 'skin', 'disorder', 'usually', 'simple', 'blood', 'test', 'identify', 'underlie', 'because.â', 'sure', 'ask', 'veterinarian', 'significant', 'change', 'dog’s', 'coat']",infection
12," Dry, Flaky Skin ","['dry', 'flaky', 'skin', 'red', 'flag', 'number', 'problem', 'common', 'symptom', 'allergy', 'mange', 'skin', 'disease', 'often', 'dry', 'flaky', 'skin', 'nothing', 'serious', 'make', 'sure', 'feed', 'fido', 'high', 'quality', 'food', 'like', 'people', 'dog', 'simply', 'get', 'dry', 'skin', 'winter', 'seem', 'pet', 'discomfort', 'consult', 'veterinarian', 'ask', 'whether', 'fatty', 'acid', 'supplementâ', 'humidifier', 'might', 'help']",
13, Acral Lick Granuloma ,"['also', 'call', 'acral', 'lick', 'dermatitis', 'frustrate', 'skin', 'condition', 'cause', 'compulsive', 'relentless', 'lick', 'single', 'area', '--', 'often', 'front', 'low', 'leg', 'area', 'unable', 'heal', 'result', 'pain', 'itch', 'lead', 'dog', 'keep', 'lick', 'spot', 'treatment', 'include', 'discourage', 'dog', 'lick', 'either', 'use', 'bad-tasting', 'topical', 'solution', 'elizabethan', 'collar', 'also', 'ask', 'dog', ""'s"", 'vet', 'treatment', 'option']",
14, Skin Tumors ,"['notice', 'lump', 'dog', ""'s"", 'skin', 'point', 'vet', 'soon', 'possible', 'dog', 'develop', 'cancerous', 'tumor', 'skin', 'way', 'confirm', 'diagnosis', 'cancer', 'biopsy', 'tumor', 'lump', 'small', 'enough', 'veterinarian', 'may', 'recommend', 'remove', 'entirely', 'yield', 'diagnosis', 'treatment', 'single', 'procedure', 'tumor', 'spread', 'may', 'treatment', 'need']",
15, Hot Spots ,"['hot', 'spot', 'also', 'call', 'acute', 'moist', 'dermatitis', 'small', 'area', 'appear', 'red', 'irritated', 'inflame', 'commonly', 'find', 'dog', ""'s"", 'head', 'hips', 'chest', 'often', 'feel', 'hot', 'touch', 'hot', 'spot', 'result', 'wide', 'range', 'condition', 'include', 'infection', 'allergy', 'insect', 'bite', 'excessive', 'lick', 'chew', 'treatment', 'consist', 'cleanse', 'hot', 'spot', 'address', 'underlying', 'condition']",infection
16, Immune Disorders ,"['rare', 'case', 'skin', 'lesion', 'infection', 'won’t', 'heal', 'indicate', 'immune', 'disorder', 'dog', 'one', 'best', 'know', 'lupus', 'disease', 'affect', 'dogs', 'people', 'lupus', 'autoimmune', 'disorder', 'meaning', 'body’s', 'immune', 'system', 'attack', 'cell', 'symptom', 'include', 'skin', 'abnormality', 'kidney', 'problem', 'fatal', 'untreated']",infection
17, Anal Sac Disease ,"['dog', 'poop', 'smelly', 'enough', 'dog', 'release', 'foul-smelling', 'substance', 'business', 'substance', 'come', 'small', 'anal', 'sac', 'become', 'impacted', 'empty', 'properly', 'hallmark', 'impact', 'anal', 'sac', 'dog', 'scoot', 'bottom', 'along', 'ground', 'symptom', 'include', 'bite', 'lick', 'anal', 'area', 'vet', 'manually', 'express', 'full', 'anal', 'sac', 'severe', 'case', 'sacs', 'may', 'surgically', 'remove']",
18, When to See the Vet ,"['although', 'skin', 'problem', 'emergencies', 'important', 'get', 'accurate', 'diagnosis', 'condition', 'treat', 'see', 'veterinarian', 'dog', 'scratch', 'lick', 'excessively', 'notice', 'change', 'pet', ""'s"", 'coat', 'skin', 'include', 'scale', 'redness', 'discoloration', 'bald', 'patch', 'identify', 'skin', 'problem', 'respond', 'well', 'treatment']",
# nltk.download('punkt')
# nltk.download('stopwords')
# nltk.download('averaged_perceptron_tagger')
# nltk.download('wordnet')
# nltk.download('omw-1.4')
# import fasttext
import string
import contractions
import nltk
import pandas as pd
from matplotlib import pyplot as plt
from nltk.corpus import stopwords, wordnet
from nltk.stem import WordNetLemmatizer
from nltk.tokenize import word_tokenize
plt.xticks(rotation=70)
pd.options.mode.chained_assignment = None
pd.set_option('display.max_colwidth', 100)
# read file
with open('items.csv') as f:
df = pd.read_csv(f)
f.close()
# check for missing values
for col in df.columns:
print(col, df[col].isnull().sum())
rws = df.loc[:, ['Name', 'Info']]
# expanding contractions “I’ve” = “I have”
rws['no_contract'] = rws['Info'].apply(lambda x: [contractions.fix(word) for word in x.split()])
rws.head()
# rejoin words and build string
rws['Info_str'] = [' '.join(map(str, l)) for l in rws['no_contract']]
rws.head()
# split each individual word into a token
rws['tokenized'] = rws['Info_str'].apply(word_tokenize)
rws.head()
# converting all characters to lowercase
rws['lower'] = rws['tokenized'].apply(lambda x: [word.lower() for word in x])
rws.head()
# removing punctuation
punc = string.punctuation
rws['no_punc'] = rws['lower'].apply(lambda x: [word for word in x if word not in punc])
rws.head()
# removing stopwords "you, he, she, in, a, has, are, etc."
stop_words = set(stopwords.words('english'))
rws['stopwords_removed'] = rws['no_punc'].apply(lambda x: [word for word in x if word not in stop_words])
rws.head()
# apply parts of speech "noun, verb, adverb, etc."
rws['pos_tags'] = rws['stopwords_removed'].apply(nltk.tag.pos_tag)
rws.head()
# NLTK’s word lemmatizer which needs the parts of speech tags to be converted to wordnet’s format
def get_wordnet_pos(tag):
if tag.startswith('J'):
return wordnet.ADJ
elif tag.startswith('V'):
return wordnet.VERB
elif tag.startswith('N'):
return wordnet.NOUN
elif tag.startswith('R'):
return wordnet.ADV
else:
return wordnet.NOUN
rws['wordnet_pos'] = rws['pos_tags'].apply(lambda x: [(word, get_wordnet_pos(pos_tag)) for (word, pos_tag) in x])
rws.head()
# apply NLTK’s word lemmatizer to get the first form of word
wnl = WordNetLemmatizer()
rws['lemmatized'] = rws['wordnet_pos'].apply(lambda x: [wnl.lemmatize(word, tag) for word, tag in x])
rws.head()
# f = lambda x: ', '.join(set([y for y in x.split() if y in s]))
# rws['Keyword1'] = rws['lemmatized'].apply(f)
# rws['Keyword1'] = rws['lemmatized'].apply(lambda x: ', '.join(set(x.split()).intersection(s)))
# rws['filtered'] = rws['lemmatized'].str.contains('allergic' or 'fungus').head()
# filter based on keywords to identify the infection type
keywords = ['allergic', 'infection', 'fungus', 'viral', 'environmental', 'flea', 'mite', 'yeast', 'allergy', 'dandruff', 'environment', 'fleas']
s = set(keywords)
rws['Infection'] = rws['Info'].str.findall('|'.join(keywords)).apply(set).str.join(', ')
# remove columns
# rws.drop('pos_tags', axis=1, inplace=True)
# rws.drop('tokenized', axis=1, inplace=True)
# rws.drop('Info_str', axis=1, inplace=True)
# rws.drop('wordnet_pos', axis=1, inplace=True)
# rws.drop('stopwords_removed', axis=1, inplace=True)
# rws.drop('no_punc', axis=1, inplace=True)
# rws.drop('lower', axis=1, inplace=True)
# rws.drop('no_contract', axis=1, inplace=True)
# rws.drop('Info', axis=1, inplace=True)
# print output to csv
rws.to_csv('pre_processed_disease_infection1.csv')
,Name,lemmatized,Infection
0, Dog Skin Problems ,"['sound', 'dog', 'constantly', 'scratch', 'lick', 'irritate', 'nail', 'chalkboard', 'don’t', 'blame', 'pooch', 'bad', 'habit', '--', 'skin', 'condition', 'probably', 'culprit', 'possible', 'cause', 'range', 'parasite', 'allergy', 'underlie', 'illness', 'webmd', 'compile', 'image', 'common', 'canine', 'skin', 'problem']",
1, Allergic Dermatitis ,"['dog', 'allergic', 'reaction', 'groom', 'product', 'food', 'environmental', 'irritant', 'pollen', 'insect', 'bite', 'dog', 'allergy', 'may', 'scratch', 'relentlessly', 'peek', 'skin', 'often', 'reveal', 'ugly', 'rash', 'corticosteroid', 'new', 'medicine', 'help', 'itchy', 'rashes.â', 'effective', 'treatment', 'identify', 'avoid', 'exposure', 'allergen']","environmental, allergic"
2, Yeast Infection ,"['dog', 'seem', 'stop', 'scratch', 'ear', 'lick', 'chewing', 'toe', 'ask', 'veterinarian', 'check', 'yeast', 'infection', 'symptom', 'include', 'irritated', 'itchy', 'discolor', 'skin', 'infection', 'usually', 'strike', 'paws', 'ear', 'yeast', 'cozy', 'space', 'grow', 'yeast', 'infection', 'easy', 'diagnose', 'often', 'respond', 'well', 'topical', 'cream', 'case', 'veterinarian', 'may', 'prescribe', 'oral', 'drug', 'medicate', 'spray', 'â', 'medicate', 'bath']","infection, yeast"
3, Folliculitis ,"['superficial', 'bacterial', 'folliculitis', 'infection', 'cause', 'sore', 'bump', 'scab', 'skin', 'skin', 'abnormality', 'easier', 'see', 'shorthaired', 'dog', 'longhaired', 'dog', 'obvious', 'symptom', 'may', 'dull', 'coat', 'shed', 'scaly', 'skin', 'underneath', 'folliculitis', 'often', 'occur', 'conjunction', 'skin', 'problem', 'mange', 'allergy', 'injury', 'treatment', 'may', 'include', 'oral', 'antibiotic', 'antibacterial', 'ointment', 'shampoo']",infection
4, Impetigo ,"['another', 'type', 'bacterial', 'infection', 'impetigo', 'common', 'puppy', 'cause', 'pus-filled', 'blister', 'may', 'break', 'crust', 'blister', 'usually', 'develop', 'hairless', 'portion', 'abdomen', 'impetigo', 'rarely', 'serious', 'treat', 'topical', 'solution', 'small', 'number', 'case', 'infection', 'may', 'spread', 'persist']",infection
5, Seborrhea ,"['seborrhea', 'cause', 'dog', ""'s"", 'skin', 'become', 'greasy', 'develop', 'scale', 'dandruff', 'case', 'genetic', 'disease', 'begin', 'dog', 'young', 'last', 'lifetime', 'dog', 'seborrhea', 'develop', 'scale', 'complication', 'another', 'medical', 'problem', 'allergy', 'hormonal', 'abnormality', 'case', 'vital', 'treat', 'underlie', 'symptom', 'recur', 'seborrhea', 'typically', 'treat', 'certain', 'medicated', 'shampoo']",dandruff
6, Ringworm ,"['despite', 'name', 'ringworm', 'cause', 'worm', 'fungus', 'term', '``', 'ring', ""''"", 'come', 'circular', 'patch', 'form', 'anywhere', 'often', 'find', 'dog', ""'s"", 'head', 'paw', 'ears', 'forelegs', 'inflammation', 'scaly', 'patch', 'hair', 'loss', 'often', 'surround', 'lesion', 'puppy', 'less', 'year', 'old', 'susceptible', 'infection', 'spread', 'quickly', 'dog', 'kennel', 'pet', 'owner', 'home', 'various', 'anti-fungal', 'treatment', 'available']","infection, fungus"
7, Shedding and Hair Loss (Alopecia) ,"['anyone', 'share', 'home', 'dog', 'know', 'would', 'much', 'shed', 'normal', 'depends', 'breed', 'time', 'year', 'environment', 'sometimes', 'stress', 'poor', 'nutrition', 'illness', 'dog', 'lose', 'hair', 'usual', 'abnormal', 'excessive', 'shed', 'persists', 'week', 'notice', 'patch', 'miss', 'fur', 'check', 'veterinarian']",environment
8, Mange (Mites) ,"['mange', 'skin', 'disorder', 'cause', 'tiny', 'parasite', 'call', 'mite']",mite
9, Fleas ,"['flea', 'bane', 'pet', 'owner', 'may', 'see', 'tiny', 'insect', 'flea', 'droppings', 'egg', 'usually', 'visible', 'dog', ""'s"", 'coat', 'symptom', 'include', 'excessive', 'lick', 'scratch', 'scabs', 'hot', 'spot', 'severe', 'flea', 'infestation', 'blood', 'loss', 'anemia', 'even', 'expose', 'dog', 'parasite', 'tapeworm', 'treatment', 'may', 'include', 'topical', 'and/or', 'oral', 'flea', 'killer', 'thorough', 'cleaning', 'pet', ""'s"", 'home', 'yard']",flea
10, Ticks ,"['tick', 'like', 'fleas', 'external', 'parasite', 'fee', 'blood', 'host', 'spot', 'tick', 'feeding', 'dog', 'naked', 'eye', 'properly', 'remove', 'tick', 'grasp', 'tick', 'tweezer', 'close', 'dog’s', 'skin', 'gently', 'pull', 'straight', 'twist', 'pull', 'hard', 'may', 'head', 'remain', 'lodged', 'dog’s', 'skin', 'lead', 'infection', 'place', 'tick', 'jar', 'alcohol', 'couple', 'day', 'pet', 'get', 'ill', 'vet', 'may', 'need', 'analyze', 'wrong.â', 'addition', 'cause', 'blood', 'loss', 'anemia', 'tick', 'transmit', 'lyme', 'disease', 'potentially', 'serious', 'bacterial', 'infection', 'live', 'area', 'tick', 'common', 'talk', 'veterinarian', 'tick', 'control', 'product']","flea, infection"
11, Color or Texture Changes ,"['change', 'dog', ""'s"", 'skin', 'color', 'coat', 'texture', 'warn', 'sign', 'several', 'common', 'metabolic', 'hormone', 'problem', 'also', 'result', 'infection', 'skin', 'disorder', 'usually', 'simple', 'blood', 'test', 'identify', 'underlie', 'because.â', 'sure', 'ask', 'veterinarian', 'significant', 'change', 'dog’s', 'coat']",infection
12," Dry, Flaky Skin ","['dry', 'flaky', 'skin', 'red', 'flag', 'number', 'problem', 'common', 'symptom', 'allergy', 'mange', 'skin', 'disease', 'often', 'dry', 'flaky', 'skin', 'nothing', 'serious', 'make', 'sure', 'feed', 'fido', 'high', 'quality', 'food', 'like', 'people', 'dog', 'simply', 'get', 'dry', 'skin', 'winter', 'seem', 'pet', 'discomfort', 'consult', 'veterinarian', 'ask', 'whether', 'fatty', 'acid', 'supplementâ', 'humidifier', 'might', 'help']",
13, Acral Lick Granuloma ,"['also', 'call', 'acral', 'lick', 'dermatitis', 'frustrate', 'skin', 'condition', 'cause', 'compulsive', 'relentless', 'lick', 'single', 'area', '--', 'often', 'front', 'low', 'leg', 'area', 'unable', 'heal', 'result', 'pain', 'itch', 'lead', 'dog', 'keep', 'lick', 'spot', 'treatment', 'include', 'discourage', 'dog', 'lick', 'either', 'use', 'bad-tasting', 'topical', 'solution', 'elizabethan', 'collar', 'also', 'ask', 'dog', ""'s"", 'vet', 'treatment', 'option']",
14, Skin Tumors ,"['notice', 'lump', 'dog', ""'s"", 'skin', 'point', 'vet', 'soon', 'possible', 'dog', 'develop', 'cancerous', 'tumor', 'skin', 'way', 'confirm', 'diagnosis', 'cancer', 'biopsy', 'tumor', 'lump', 'small', 'enough', 'veterinarian', 'may', 'recommend', 'remove', 'entirely', 'yield', 'diagnosis', 'treatment', 'single', 'procedure', 'tumor', 'spread', 'may', 'treatment', 'need']",
15, Hot Spots ,"['hot', 'spot', 'also', 'call', 'acute', 'moist', 'dermatitis', 'small', 'area', 'appear', 'red', 'irritated', 'inflame', 'commonly', 'find', 'dog', ""'s"", 'head', 'hips', 'chest', 'often', 'feel', 'hot', 'touch', 'hot', 'spot', 'result', 'wide', 'range', 'condition', 'include', 'infection', 'allergy', 'insect', 'bite', 'excessive', 'lick', 'chew', 'treatment', 'consist', 'cleanse', 'hot', 'spot', 'address', 'underlying', 'condition']",infection
16, Immune Disorders ,"['rare', 'case', 'skin', 'lesion', 'infection', 'won’t', 'heal', 'indicate', 'immune', 'disorder', 'dog', 'one', 'best', 'know', 'lupus', 'disease', 'affect', 'dogs', 'people', 'lupus', 'autoimmune', 'disorder', 'meaning', 'body’s', 'immune', 'system', 'attack', 'cell', 'symptom', 'include', 'skin', 'abnormality', 'kidney', 'problem', 'fatal', 'untreated']",infection
17, Anal Sac Disease ,"['dog', 'poop', 'smelly', 'enough', 'dog', 'release', 'foul-smelling', 'substance', 'business', 'substance', 'come', 'small', 'anal', 'sac', 'become', 'impacted', 'empty', 'properly', 'hallmark', 'impact', 'anal', 'sac', 'dog', 'scoot', 'bottom', 'along', 'ground', 'symptom', 'include', 'bite', 'lick', 'anal', 'area', 'vet', 'manually', 'express', 'full', 'anal', 'sac', 'severe', 'case', 'sacs', 'may', 'surgically', 'remove']",
18, When to See the Vet ,"['although', 'skin', 'problem', 'emergencies', 'important', 'get', 'accurate', 'diagnosis', 'condition', 'treat', 'see', 'veterinarian', 'dog', 'scratch', 'lick', 'excessively', 'notice', 'change', 'pet', ""'s"", 'coat', 'skin', 'include', 'scale', 'redness', 'discoloration', 'bald', 'patch', 'identify', 'skin', 'problem', 'respond', 'well', 'treatment']",
This source diff could not be displayed because it is too large. You can view the blob instead.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment