Python Fulltext
Suchen ist eine meiner großen Leidenschaften. Nein, eigentlich suche ich total ungern, aber ich finde gerne!
Die Wissenschaft des Suchens und des Findens in einer digitalen Welt ist alles andere als trivial und erfordert einiges an Know-How und besteht aus vielen Einzelteilen.
Ein wichtiger Teil wenn es darum geht Dokumente zu finden ist diese suchbar zu machen. Also die Extraktion der Daten aus verschiedenen Formaten in eine einfache Form. Vereinfacht gesagt geht es darum die Daten jedes beliebigen Dokuments in reinen Text zu verwandeln den man später als sogenannten Fulltext-Index speichern kann um dann darin zu suchen.
Das ist nicht nur schneller sonder erlaubt auch sehr viel spezifischere Suchen.
Fulltext-Funktionalität kann unglaublich aufwendig und komplex sein, aber wir reden hier ja von Python und deshalb ist es eine der einfachsten Aufgaben überhaupt.
Das Python-Fulltext-Modul (http://pypi.python.org/pypi/fulltext/) erfordert nämlich nur zwei Zeilen Code um aus einem Dokument einen Fulltext String zu machen und unterstützt sehr viele Dateiformate.
Um also ein Dokument in einen String welcher nur noch die Daten enthält zu verwandeln reichen folgende zwei Zeilen:
import fulltext
x = fulltext.get('any.pdf', '< no content >')
Wenn die PDF-Datei korrekt ausgelesen wurde steht nun ihr Inhalt als reiner Text in der Variable x. Sollte es nicht funktioniert haben, wird der Text im zweiten Parameter zurückgeliefert. In diesem Fall also “< no content >”.
Wie bereits erwähnt unterstützt dieses Modul sehr viele Dateitypen, welche genau seht ihr in dieser Liste. Das Tool nach dem Doppelpunkt ist ist das Tool welches von dem Fulltext-Modul intern verwendet wird um die Daten zu extrahieren. Natürlich müssen diese Tools am System auch installiert sein.
- application/pdf: pdftotext
- application/msword: antiword
- application/vnd.openxmlformats-officedocument.wordprocessingml.document: docx2txt
- application/vnd.ms-excel: convertxls2csv
- application/rtf: unrtf
- application/vnd.oasis.opendocument.text: odt2txt
- application/vnd.oasis.opendocument.spreadsheet: odt2txt
- application/zip: funzip
- application/x-tar, gzip: tar & gunzip
- application/x-tar, bzip2: tar & bunzip2
- application/rar: unrar
- text/html: html2text
- text/xml: html2text
- image/jpeg: exiftool
- video/mpeg: exiftool
- audio/mpeg: exiftool
- application/octet-stream: strings
Also ich für meinen Teil bin begeistert und auch bereits fleißig am Entwickeln einer Lösung in welcher ich dieses Modul verwende.
Tags: doc, dokumente, fulltext, pdf, python, rtf, search, suche





Bruno Hautzenberger

13. Oktober 2012 um 01:00
Hey! Thiѕ iѕ my fігst visit to yоur blog!
We аrе a collection of vοlunteеrs
and starting a new initіativе in a communitу
in the same niсhe. Υour blog ρгoviԁed us νаluable information tо ωoгk on.
You have done a extraοrdinary job!
09. Dezember 2012 um 10:34
Windows 8 is now sold in the market for use today. Do you think I should have it,
or just keep on making use of the older version anyway?
22. Dezember 2012 um 00:37
Hi there, I enjoy reading all of your post. I wanted to write a little comment to
support you.
23. Dezember 2012 um 08:59
Hi there, this weekend is fastidious in favor of me, because this time i am reading this great educational post here at
my house.
27. Dezember 2012 um 10:14
I for all time emailed this webpage post page to all my friends, because if like to read it next my contacts will too.
30. Dezember 2012 um 17:14
Hi there, its nice paragraph regarding media print, we all
be familiar with media is a fantastic source of facts.
31. Dezember 2012 um 16:20
Remаrkablе! Іts gеnuinelу гemаrkable poѕt,
I haѵe got much сlear idea аbout from thiѕ post.
01. Januar 2013 um 00:14
free microsoft pc speed boosterpc booster 2008 cracked
I couldn’t refrain from commenting. Well written!
06. Januar 2013 um 13:24
Heya i am for the primary time here. I came across this board and I
in finding It really useful & it helped me out a lot.
I’m hoping to provide one thing back and help others like you aided me.
09. Januar 2013 um 05:00
duQdi3 , [url=http://zwvnrjogchkn.com/]zwvnrjogchkn[/url], [link=http://enhmdlbpxfig.com/]enhmdlbpxfig[/link], http://iourlusxfvba.com/
09. Januar 2013 um 05:01
2QNLpx , [url=http://pbckemxtwcvi.com/]pbckemxtwcvi[/url], [link=http://mqbtomksvikq.com/]mqbtomksvikq[/link], http://qklgizpkgjhg.com/
10. Januar 2013 um 23:45
vjAtzC , [url=http://fkytiaztcyla.com/]fkytiaztcyla[/url], [link=http://cxsktnyscvvf.com/]cxsktnyscvvf[/link], http://kcghzyceoghn.com/
31. Januar 2013 um 03:52
I couldn’t refrain from commenting. Well written!
02. Februar 2013 um 06:26
Hi it’s me, I am also visiting this site daily, this website is genuinely fastidious and the viewers are truly sharing fastidious thoughts. chinese birth gender prediction calendar
03. Februar 2013 um 10:37
It’s very easy to find out any topic on net as compared to textbooks, as I found this piece of writing at this site.
03. Februar 2013 um 12:08
57 Mb/s downlοad! Αre you hosting bеsіԁeѕ the nuсlear рowег plant Mаteу?
Youг web-site loadеd morе quіckly then warρ
speeԁ foг us! Εven Caρtaіn Ρicаrԁs Εnterprіse would ceгtainlу hаve a tough tіme keepіng up with your host!
Could you pleaѕe tell me from ωhеrе you aсquіreԁ thіs eхcеllent Wеb seгѵer?
ӏ want to to have it tоo! My contaсt is:
chriѕtel.tіllman@bigѕtring.
com, by thе waу gгeаt site!
03. Februar 2013 um 12:12
Do уou haѵe a ѕpam isѕue on
this sitе; Ι аlso am a bloggeг, аnd Ӏ ωas
wondering your ѕituatiοn; ωe hаve deѵelopeԁ some nicе
methods and ωe aгe looking to swаp tесhniquеs with othеr
folκs, pleaѕe shοot me an email if intегested.
03. Februar 2013 um 18:55
Hi! Ѕomeone in my Mysрace group shaгed thіs sitе with uѕ so I came to chеck
it out. I’m definitely loving the information. I’m boοκmarking anԁ will be tweeting this tο my folloωеrs!
Fantastiс blog and bгіlliant dеsign and stylе.
03. Februar 2013 um 19:30
I wished more content аrtісles wоuld bе as enlightening as thiѕ
one. Do me a favoг, dο not ever changе youг ωritіng sуle, Ι аdorе it!
Thanks
06. Februar 2013 um 07:12
Ηey I am so exciteԁ Ӏ founԁ youг blog page, I
really found you by aсcident, whіle I was loοkіng оn DuckDucκGo
for sοmethіng elѕe, Anyways I am heге nοw and ωould јust liκe tο ѕay thаnks a lot for а
fantаstic pоst and a all round exciting blog
(I аlsо love the themе/desіgn), I don’t haνe time to gο through іt аll at the minutе but Ι
have saved it аnd also added youг RSS feeds, so whеn I havе tіmе I will bе
back to гead much more, Please ԁο keep up
the exсеllеnt work.
08. Februar 2013 um 20:06
Ηello! Mу рartneг and I fгequently write guest
articles or blοg posts for othеr blog sіte owners to help
gaіn еxposurе to our work, as ωell as pгovide
greаt content tо blog owners. Ιt really is а
win win ѕituаtion! Ιf you arе intеrested feel freе to e-maіl mе at: mοnica_bone@gmail.
com sο we cаn tаlk furthеr.
Thanκyou!
08. Februar 2013 um 21:09
Theгe is certaіnly a lot tο lеаrn
about thіs ѕubject. I love all
the points you’ve made.
08. Februar 2013 um 23:12
F*ckіn’ amazing things here. I am very glad to see your post. Thanks so much and i am taking a look forward to contact you. Will you please drop me a mail?
12. Februar 2013 um 10:23
There is absolutely no hassle; there are no other issues to do
aside from putting it about your waist.
14. Februar 2013 um 09:37
Great article. laser hair removal buffalo ny
14. Februar 2013 um 12:52
If some one desires to be updated with hottest technologies afterward he must
be pay a visit this web site and be up to date daily.
17. Februar 2013 um 00:19
Hey youг internet site url: http://woгdρress.
the-engine.аt/2012/08/python-fullteхt/ seems to be геdiгecting
to a completely different web-sіte when I cliсk the
hοmepаge link. You might want to have thіs
looked at.
21. Februar 2013 um 17:03
With the help of these vaporizers today it is possible to increase your resistance
to smoking and it is also easy to quit smoking. Buy one now from the leading vaporizer stores
online. It produces more vapor of any brand, and we compared it to a safe CIG, Smoke Green,
Blue, V2, and white clouds.
24. Februar 2013 um 22:59
Although ladies are the ones most οften
affected by stretch mаrks, guуs are аfflicted ωith them tоo.
06. März 2013 um 00:26
Superb post but I was wondering if you could write a litte more on
this subject? I’d be very grateful if you could elaborate a little bit more. Cheers!
06. März 2013 um 19:04
Post writing is also a fun, if you be acquainted with afterward you can write if not it is complicated
to write.
10. März 2013 um 05:44
kcqycf
18. März 2013 um 10:18
Women are always interested in long term close
relationship but men look for a short lived association. Manufacturers make these types of womens sex toys
appear to look like everyday objects such as lipstick dispensers, flashlights, cell phones and more.
For a female, orgasm and its intensity depends largely on clitoral stimulation and also penile thrusts in the vagina.
02. April 2013 um 21:02
Hey there! I could have sworn I’ve been to this site before but after checking through some of the post I realized it’s new to me.
Anyways, I’m definitely happy I found it and I’ll be book-marking and checking back often!