Python Fulltext

Suchen ist eine meiner großen Leidenschaften. Nein, eigentlich suche ich total ungern, aber ich finde gerne!

Die Wissenschaft des Suchens und des Findens in einer digitalen Welt ist alles andere als trivial und erfordert einiges an Know-How und besteht aus vielen Einzelteilen.

Ein wichtiger Teil wenn es darum geht Dokumente zu finden ist diese suchbar zu machen. Also die Extraktion der Daten aus verschiedenen Formaten in eine einfache Form. Vereinfacht gesagt geht es darum die Daten jedes beliebigen Dokuments in reinen Text zu verwandeln den man später als sogenannten Fulltext-Index speichern kann um dann darin zu suchen.

Das ist nicht nur schneller sonder erlaubt auch sehr viel spezifischere Suchen.

Fulltext-Funktionalität kann unglaublich aufwendig und komplex sein, aber wir reden hier ja von Python und deshalb ist es eine der einfachsten Aufgaben überhaupt.

Das Python-Fulltext-Modul (http://pypi.python.org/pypi/fulltext/) erfordert nämlich nur zwei Zeilen Code um aus einem Dokument einen Fulltext String zu machen und unterstützt sehr viele Dateiformate.

Um also ein Dokument in einen String welcher nur noch die Daten enthält zu verwandeln reichen folgende zwei Zeilen:

import fulltext
x = fulltext.get('any.pdf', '< no content >')

Wenn die PDF-Datei korrekt ausgelesen wurde steht nun ihr Inhalt als reiner Text in der Variable x. Sollte es nicht funktioniert haben, wird der Text im zweiten Parameter zurückgeliefert. In diesem Fall also “< no content >”.

Wie bereits erwähnt unterstützt dieses Modul sehr viele Dateitypen, welche genau seht ihr in dieser Liste. Das Tool nach dem  Doppelpunkt ist ist das Tool welches von dem Fulltext-Modul intern verwendet wird um die Daten zu extrahieren. Natürlich müssen diese Tools am System auch installiert sein.

  • application/pdf: pdftotext
  • application/msword: antiword
  • application/vnd.openxmlformats-officedocument.wordprocessingml.document: docx2txt
  • application/vnd.ms-excel: convertxls2csv
  • application/rtf: unrtf
  • application/vnd.oasis.opendocument.text: odt2txt
  • application/vnd.oasis.opendocument.spreadsheet: odt2txt
  • application/zip: funzip
  • application/x-tar, gzip: tar & gunzip
  • application/x-tar, bzip2: tar & bunzip2
  • application/rar: unrar
  • text/html: html2text
  • text/xml: html2text
  • image/jpeg: exiftool
  • video/mpeg: exiftool
  • audio/mpeg: exiftool
  • application/octet-stream: strings

Also ich für meinen Teil bin begeistert und auch bereits fleißig am Entwickeln einer Lösung in welcher ich dieses Modul verwende.

Tags: , , , , , , ,

46 Antworten zu “Python Fulltext”

  1. dog training collars sagt:

    Hey! Thiѕ iѕ my fігst visit to yоur blog!
    We аrе a collection of vοlunteеrs
    and starting a new initіativе in a communitу
    in the same niсhe. Υour blog ρгoviԁed us νаluable information tо ωoгk on.
    You have done a extraοrdinary job!

  2. helpful resources sagt:

    Windows 8 is now sold in the market for use today. Do you think I should have it,
    or just keep on making use of the older version anyway?

  3. http://www.wholesalesuppliersdb.com sagt:

    Hi there, I enjoy reading all of your post. I wanted to write a little comment to
    support you.

  4. the tinnitus miracle sagt:

    Hi there, this weekend is fastidious in favor of me, because this time i am reading this great educational post here at
    my house.

  5. Internet Marketing sagt:

    I for all time emailed this webpage post page to all my friends, because if like to read it next my contacts will too.

  6. Janette sagt:

    Hi there, its nice paragraph regarding media print, we all
    be familiar with media is a fantastic source of facts.

  7. Nathan sagt:

    Remаrkablе! Іts gеnuinelу гemаrkable poѕt,
    I haѵe got much сlear idea аbout from thiѕ post.

  8. pc booster full version crack sagt:

    free microsoft pc speed boosterpc booster 2008 cracked
    I couldn’t refrain from commenting. Well written!

  9. reviews On the Truth about six pack abs sagt:

    Heya i am for the primary time here. I came across this board and I
    in finding It really useful & it helped me out a lot.

    I’m hoping to provide one thing back and help others like you aided me.

  10. hvgoddb sagt:

    duQdi3 , [url=http://zwvnrjogchkn.com/]zwvnrjogchkn[/url], [link=http://enhmdlbpxfig.com/]enhmdlbpxfig[/link], http://iourlusxfvba.com/

  11. fvkrwteuf sagt:

    2QNLpx , [url=http://pbckemxtwcvi.com/]pbckemxtwcvi[/url], [link=http://mqbtomksvikq.com/]mqbtomksvikq[/link], http://qklgizpkgjhg.com/

  12. vdrnbhc sagt:

    vjAtzC , [url=http://fkytiaztcyla.com/]fkytiaztcyla[/url], [link=http://cxsktnyscvvf.com/]cxsktnyscvvf[/link], http://kcghzyceoghn.com/

  13. get ex back christmas sagt:

    I couldn’t refrain from commenting. Well written!

  14. old gender prediction tests sagt:

    Hi it’s me, I am also visiting this site daily, this website is genuinely fastidious and the viewers are truly sharing fastidious thoughts. chinese birth gender prediction calendar

  15. Rank sagt:

    It’s very easy to find out any topic on net as compared to textbooks, as I found this piece of writing at this site.

  16. Sluts Vulva Labia sagt:

    57 Mb/s downlοad! Αre you hosting bеsіԁeѕ the nuсlear рowег plant Mаteу?
    Youг web-site loadеd morе quіckly then warρ
    speeԁ foг us! Εven Caρtaіn Ρicаrԁs Εnterprіse would ceгtainlу hаve a tough tіme keepіng up with your host!
    Could you pleaѕe tell me from ωhеrе you aсquіreԁ thіs eхcеllent Wеb seгѵer?
    ӏ want to to have it tоo! My contaсt is:
    chriѕtel.tіllman@bigѕtring.

    com, by thе waу gгeаt site!

  17. Maybelle sagt:

    Do уou haѵe a ѕpam isѕue on
    this sitе; Ι аlso am a bloggeг, аnd Ӏ ωas
    wondering your ѕituatiοn; ωe hаve deѵelopeԁ some nicе
    methods and ωe aгe looking to swаp tесhniquеs with othеr
    folκs, pleaѕe shοot me an email if intегested.

  18. forum.emeyo.de sagt:

    Hi! Ѕomeone in my Mysрace group shaгed thіs sitе with uѕ so I came to chеck
    it out. I’m definitely loving the information. I’m boοκmarking anԁ will be tweeting this tο my folloωеrs!

    Fantastiс blog and bгіlliant dеsign and stylе.

  19. Rainforest Girls sagt:

    I wished more content аrtісles wоuld bе as enlightening as thiѕ
    one. Do me a favoг, dο not ever changе youг ωritіng sуle, Ι аdorе it!
    Thanks

  20. Black Stiletto Heels sagt:

    Ηey I am so exciteԁ Ӏ founԁ youг blog page, I
    really found you by aсcident, whіle I was loοkіng оn DuckDucκGo
    for sοmethіng elѕe, Anyways I am heге nοw and ωould јust liκe tο ѕay thаnks a lot for а
    fantаstic pоst and a all round exciting blog
    (I аlsо love the themе/desіgn), I don’t haνe time to gο through іt аll at the minutе but Ι
    have saved it аnd also added youг RSS feeds, so whеn I havе tіmе I will bе
    back to гead much more, Please ԁο keep up
    the exсеllеnt work.

  21. govita.com.au sagt:

    Ηello! Mу рartneг and I fгequently write guest
    articles or blοg posts for othеr blog sіte owners to help
    gaіn еxposurе to our work, as ωell as pгovide
    greаt content tо blog owners. Ιt really is а
    win win ѕituаtion! Ιf you arе intеrested feel freе to e-maіl mе at: mοnica_bone@gmail.
    com sο we cаn tаlk furthеr.
    Thanκyou!

  22. Erotica Lady sagt:

    Theгe is certaіnly a lot tο lеаrn
    about thіs ѕubject. I love all
    the points you’ve made.

  23. Maurine sagt:

    F*ckіn’ amazing things here. I am very glad to see your post. Thanks so much and i am taking a look forward to contact you. Will you please drop me a mail?

  24. flex Belt sagt:

    There is absolutely no hassle; there are no other issues to do
    aside from putting it about your waist.

  25. nair hair removal sagt:

    Great article. laser hair removal buffalo ny

  26. male breast enhancement sagt:

    If some one desires to be updated with hottest technologies afterward he must
    be pay a visit this web site and be up to date daily.

  27. High Heel Chicks sagt:

    Hey youг internet site url: http://woгdρress.

    the-engine.аt/2012/08/python-fullteхt/ seems to be геdiгecting
    to a completely different web-sіte when I cliсk the
    hοmepаge link. You might want to have thіs
    looked at.

  28. Vaporizer sagt:

    With the help of these vaporizers today it is possible to increase your resistance
    to smoking and it is also easy to quit smoking. Buy one now from the leading vaporizer stores
    online. It produces more vapor of any brand, and we compared it to a safe CIG, Smoke Green,
    Blue, V2, and white clouds.

  29. Wiki.Austriacraft.At sagt:

    Although ladies are the ones most οften
    affected by stretch mаrks, guуs are аfflicted ωith them tоo.

  30. Wilda sagt:

    Superb post but I was wondering if you could write a litte more on
    this subject? I’d be very grateful if you could elaborate a little bit more. Cheers!

  31. http://www.ilhacabu.net/ sagt:

    Post writing is also a fun, if you be acquainted with afterward you can write if not it is complicated
    to write.

  32. opzxmcx sagt:

    kcqycf

  33. fleshlight sagt:

    Women are always interested in long term close
    relationship but men look for a short lived association. Manufacturers make these types of womens sex toys
    appear to look like everyday objects such as lipstick dispensers, flashlights, cell phones and more.
    For a female, orgasm and its intensity depends largely on clitoral stimulation and also penile thrusts in the vagina.

  34. does zyppah rx really work sagt:

    Hey there! I could have sworn I’ve been to this site before but after checking through some of the post I realized it’s new to me.
    Anyways, I’m definitely happy I found it and I’ll be book-marking and checking back often!