SyntaxHighlighter

SyntaxHighlighter

Tuesday, July 24, 2012

Shingling in Python

As a short companion piece to my post about Shingling in XQuery, and as an exercise to keep my Python skills sharp, I rewrote my XQuery example in Python.

Character Shingling in Python

It is slightly easier to find examples of shingling in Python. For example, I found this one line example of character shingling in this blog post http://jutopia.tirsen.com/2011/08/26/shingles.html


[word[i:i + n] for i in range(len(word) - n + 1)]



This makes use of Python's list comprehension technique to succinctly render n-shingles for a given word. I used that as inspiration for my word-based w-shingle method.

Word Shingling in Python

As before, I am only interested in shingles that start with what look like stop words (approximated as being words consisting of fewer than 4 characters).

theString = "the quick brown fox jumps over the lazy dog. now is the time for all good men to come to the aid of the party"
shingleLength = 3
tokens = theString.split()


print [tokens[i:i+shingleLength] for i in range(len(tokens) - shingleLength + 1) if len(tokens[i]) < 4]



Enjoy!

1 comment:

  1. I'M TOTALLY FREE FROM HERPES VIRUS
    Herpes is a serious and recurrent condition that cannot be cured by drugs or injections by DR JAMES , but the best way to fight against herpes is to take natural herbal remedies, I red about DR JAMES, the great herbalist who cures people of HIV virus with his powerful herbal medicine. I contacted him to find out how he could help me and he told me never to be afraid that he would help me with the natural herbs medicine! After 2 days of contacting him he told me the medicine is ready and he sent it to me via DHL COURIER SERVICE and it got me in 3 days! I used the medication as he prescribed for me (MORNING and EVENING) and was cured! It's really like a dream, but I'm so happy! For people suffering from the following diseases,Shingles,MS,Bullous Pemphigoid,Diabetes, cancer,Pcos, hypothyroidism, Herpes, COPD, HIV, arthritis, Hpv, infections, liver disease, autoimmune diseases, Parkinson's disease, Lupus and more should contact him for his herbal medicine, because I am a living witness and I was cured of herpes virus. and DR James medicine is legitimate. I sent him what he asked for and he sent me his medication which I took for 2 weeks and today I am here with a negative result. When I went for the test, I was so happy after I took his herbal mix medicine.CONTACT DR JAMES FOR A PERMANENT CURE Email: drjamesherbalmix@gmail.com

    ReplyDelete