2007/08/18
Desktop search with Thunderbird and Beagle
Desktop search has quickly become an essential part of modern computing environments. For Windows operating systems there are a couple of alternatives. On Linux there's only one option: Beagle. Beagle is a reasonably mature desktop search engine, but it lacks support for Thunderbird. Fortunately this is improving. In this article I explore the current state of the art.
This was tested on Ubuntu Feisty (7.04).
Ubuntu: great, but how do I find my e-mail?
In October last year I switched to Ubuntu Linux as my primary operating system. Setting up and using Ubuntu has been a great experience. Most things worked right away; a lot of stuff requires a bit of fiddling to accomplish but in the end my system was much more pleasant to use than my Windows system. Except for one thing: desktop search.
Beagle: Linux desktop search
Enter Beagle.
Fortunately a fellow named Pierre Östlund took on the task of rewriting the integration between Beagle and Thunderbird. This has taken the form of a Thunderbird extension to export the data to a Beagle index, which the Thunderbird backend for Beagle parses fast. This is highly beta, there's no release version yet.
Download and install the extension to index your mail.
One thing left to do that's not in the manual. Install automake 1.9:
Now run autogen.sh:
Then let it roll:
When I compile stuff from source I like to use
For now we'll do it the not-so-neat-way:
Indexing your stuff
Now Beagle is installed, the Thunderbird extension can be put to work. There should be a menu option Tools > Beagle indexing settings. Enable indexing and set the speed to Very fast, this will make it process your archive fast. It went through my 41000 e-mails in about 15 minutes.
Now start the beagle daemon. There's an option documented in the FAQ which makes it index at maximum speed. Enable this to index your complete archive on the first run. Note that this will keep your PC very busy, possibly for hours if you also configured Beagle to index many folders with documents.
There's another nice option though: Gnome contains a nifty tool called the Deskbar, which can be used to start Beagle but also to start applications, start writing an e-mail by typing the name of a person in your address book, etc. It's a bit like Quicksilver and AppRocket. Not nearly as neat, but pretty functional.
The overall experience of using Beagle could be greatly improved by improving the usability, functionality and overall sexiness of the Beagle search tool. The technical core is well designed, but a great user experience is required to make end users benefit from it.
This was tested on Ubuntu Feisty (7.04).
Desktop search
My first experience with desktop search was Copernic Desktop search on Windows. I loved it from the first time I used it: from now on I could search through all my private information as easily as searching the web! Access to the right documents, e-mails, music... within seconds.Ubuntu: great, but how do I find my e-mail?
In October last year I switched to Ubuntu Linux as my primary operating system. Setting up and using Ubuntu has been a great experience. Most things worked right away; a lot of stuff requires a bit of fiddling to accomplish but in the end my system was much more pleasant to use than my Windows system. Except for one thing: desktop search.Beagle: Linux desktop search
Enter Beagle."Beagle is a search tool that ransacks your personal information space to find whatever you're looking for. More technically, Beagle is a Linux desktop-independent service which transparently and unobtrusively indexes your data in real-time."Ergo, a desktop search solution for Linux, and from the description it's a pretty good one too. It lacked one essential feature for me however: it can't search Thunderbird messages. Integration with Thunderbird was removed a while ago.
Fortunately a fellow named Pierre Östlund took on the task of rewriting the integration between Beagle and Thunderbird. This has taken the form of a Thunderbird extension to export the data to a Beagle index, which the Thunderbird backend for Beagle parses fast. This is highly beta, there's no release version yet.
Setting up Beagle to search Thunderbird e-mails
Setting it up takes four steps:- Install Thunderbird 2.0, which is not in Feisty by default, but required by the extension.
- Install the Thunderbird extension.
- Build and install Beagle from the latest SVN.
- Let them index your stuff.
Installing Thunderbird 2.0 on Feisty
Backup your profile data, it's in ~/.mozilla-thunderbird. Then follow the instructions to install TB 2.0 on help.ubuntu.com. Use the preferred method with the third-party repository.Download and install the extension to index your mail.
Building Beagle from source
I followed the installation guide. First, uninstall beagle if it's currently installed.$ sudo aptitude remove beagle$ svn checkout http://svn.gnome.org/svn/beagle/trunk/beagle
One thing left to do that's not in the manual. Install automake 1.9:
$ sudo aptitude install automake1.9REQUIRED_AUTOMAKE_VERSION to 1.9. This will prevent a critical problem with the sequence of compilation.Now run autogen.sh:
$ ./autogen.shThen let it roll:
$ makebeagled, beagle-status, etc. in the beagle folder.When I compile stuff from source I like to use
checkinstall to install the package. This creates a neat .deb and installs it in the software repository like any other package, allowing for easy uninstallation. This failed however. I'm not an expert in checkinstall; tips on how to get that to work will be appreciated.For now we'll do it the not-so-neat-way:
The binaries are installed and we're ready to roll.$ make install
Indexing your stuff
Now Beagle is installed, the Thunderbird extension can be put to work. There should be a menu option Tools > Beagle indexing settings. Enable indexing and set the speed to Very fast, this will make it process your archive fast. It went through my 41000 e-mails in about 15 minutes.Now start the beagle daemon. There's an option documented in the FAQ which makes it index at maximum speed. Enable this to index your complete archive on the first run. Note that this will keep your PC very busy, possibly for hours if you also configured Beagle to index many folders with documents.
You can check how many Thunderbird messages it has left with the following command:$ beagle-shutdown
$ export BEAGLE_EXERCISE_THE_DOG=1
$ beagled
This took under an hour on my machine. Indexing is finished, now let's search!$ ls ~/.beagle/Indexes/ThunderbirdIndex/ToIndex/ | wc
Start searching
The beagle search tool is started with thebeagle-search command or from the Applications menu: Accessoiries > Search.There's another nice option though: Gnome contains a nifty tool called the Deskbar, which can be used to start Beagle but also to start applications, start writing an e-mail by typing the name of a person in your address book, etc. It's a bit like Quicksilver and AppRocket. Not nearly as neat, but pretty functional.
My experiences
The good:- Searching is blazingly fast.
- It finds some good results.
- Double clicking on a search result opens the right message in Thunderbird, but only if Thunderbird is not currently running.
- Double clicking on a search result opens an empty message in Thunderbird if Thunderbird is already running.
- It doesn't find all results it should, and many times it finds nothing. For example the query "triathlon holten" in the screenshot above gives 27 matches in Beagle search (no hidden matches). The same query in Thunderbird (subject or body match) gives 41 matches. I get similar results for other search terms; one example term gives 3 matches in Beagle and almost a hundred in Thunderbird.
- The Beagle search interface isn't very good.
- It doesn't show the contents of the e-mails it finds. For other results, for example documents, it shows only a single line. It would be ideal to see the entire e-mail and document right away, but it should show at least a couple of lines in plain text.
- It shows results from different sources in a fixed sequence, which isn't customizable. E-mail results always end up at the bottom; I'd always want them at the top because they're usually all I need.
Conclusion
The basics of the new Thunderbird extension and Beagle backend work well. Although it's unfinished software and setting it up requires installing development packages and compiling sources, this is relatively easy to do successfully. There are some bugs left to squash; I'm confident this will happen in the near future and we'll see a stable version enter major distributions. This is a great step forward for Linux desktop search.The overall experience of using Beagle could be greatly improved by improving the usability, functionality and overall sexiness of the Beagle search tool. The technical core is well designed, but a great user experience is required to make end users benefit from it.
Labels: beagle, desktop search, thunderbird, ubuntu
2006/11/13
Caps lock = Back space
What's the last time you really used your Caps lock key? And how many times since that time did you hit it by accident, giving you more letters in capitals than you ever wanted? For me the answers were "probably never" and "too many to count". This was a reason to take action.
I spend a lot of time using a computer. Some time ago it occurred to me I should do everything possible to make the way I use computers as pleasant as possible. One thing I did was switch to the Dvorak keyboard layout. Another is typing on a Microsoft Natural Keyboard -- it saves my hands from taking on really awkward positions while typing. The new 4000 model removed the two real flaws the previous models had: 1. the front side is elevated so your wrists bend down slightly instead of up (which is supposed to be BAD, BAD, BAD!) (hey, that's a phrase I could have used my caps lock with!). And 2. the Insert/Home/Delete/End/PgUp/PgDn keys are where they should be.
So when the thought about the Caps lock key popped up, I wouldn't rest until I had improved the situation. One option would be to disable the key altogether. Not very nice for a key in such a prominent place. Another (commonly used, as I found out) option is to let it function as a Control key. Since my keyboard already has two of those, also not very useful. A better use for the Caps lock key is to let it be a Backspace. The Backspace key is one I use a lot... I want to type fast, and usually I make a lot of typos so I'm constantly hitting Backspace. And the Backspace key is not in a good place to use frequently, my right hand either has to move a whole lot to the right (not good), or bend to the right (even worse) to hit it.
This is how you go about to make your Caps lock key into a Backspace. In Windows, save the following bit to a file named "capsbackspace.reg", double-click it as an Administrator user and reboot. The change is system-wide, so anyone else using your computer might be surprised when they WANT TO WRITE AN EMAIL IN ALL CAPS.
These days I use Ubuntu Linux on my primary system. In a desktop Linux version running X, the following two lines will do the trick. Add them to a small shell script and have this started everytime you log on.
Under Linux in text mode it's a bit more complex. This worked for me in Debian:
Where it says Caps_Lock, replace it with Control_H or BackSpace. Save the file.
The keymap is now loaded, test whether it works. If it does, reboot to verify whether the mapping is indeed applied on every boot.
Apparently I'm not alone in this. Join the crusade and free yourself of Caps lock! :)
I spend a lot of time using a computer. Some time ago it occurred to me I should do everything possible to make the way I use computers as pleasant as possible. One thing I did was switch to the Dvorak keyboard layout. Another is typing on a Microsoft Natural Keyboard -- it saves my hands from taking on really awkward positions while typing. The new 4000 model removed the two real flaws the previous models had: 1. the front side is elevated so your wrists bend down slightly instead of up (which is supposed to be BAD, BAD, BAD!) (hey, that's a phrase I could have used my caps lock with!). And 2. the Insert/Home/Delete/End/PgUp/PgDn keys are where they should be.
So when the thought about the Caps lock key popped up, I wouldn't rest until I had improved the situation. One option would be to disable the key altogether. Not very nice for a key in such a prominent place. Another (commonly used, as I found out) option is to let it function as a Control key. Since my keyboard already has two of those, also not very useful. A better use for the Caps lock key is to let it be a Backspace. The Backspace key is one I use a lot... I want to type fast, and usually I make a lot of typos so I'm constantly hitting Backspace. And the Backspace key is not in a good place to use frequently, my right hand either has to move a whole lot to the right (not good), or bend to the right (even worse) to hit it.
This is how you go about to make your Caps lock key into a Backspace. In Windows, save the following bit to a file named "capsbackspace.reg", double-click it as an Administrator user and reboot. The change is system-wide, so anyone else using your computer might be surprised when they WANT TO WRITE AN EMAIL IN ALL CAPS.
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Keyboard Layout]
"Scancode Map"=hex:00,00,00,00,00,00,00,00,02,00,00,00,0e,00,3a,00,00,00,00,00
These days I use Ubuntu Linux on my primary system. In a desktop Linux version running X, the following two lines will do the trick. Add them to a small shell script and have this started everytime you log on.
# Remove bindings for Caps lock function
xmodmap -e "clear Lock"
# Add binding from Caps lock key to Backspace function
xmodmap -e "keysym Caps_Lock = BackSpace"
Under Linux in text mode it's a bit more complex. This worked for me in Debian:
cd /etc/console/
gunzip boottime.kmap.gz
joe boottime.kmap
Where it says Caps_Lock, replace it with Control_H or BackSpace. Save the file.
gzip boottime.kmap
loadkeys boottime.kmap.gz
The keymap is now loaded, test whether it works. If it does, reboot to verify whether the mapping is indeed applied on every boot.
Apparently I'm not alone in this. Join the crusade and free yourself of Caps lock! :)
Let your mobile lead the way
Lately I've been looking for nice software for my SonyEricsson K700i. It's not a high-end phone, but it runs Java apps and it has a nice color screen. So I figured there must be cool stuff out there. And there is! A good site to start is GetJar. I found Wayfinder Earth there, downloaded it and sent it to the phone via Bluetooth. Unfortunately it didn't work -- the application would start, but end up in an endless "Loading..." screen. Then I found out it wasn't supposed to work on the K700i. Bummer.
So you can imagine how happy I was to find out that Google Maps has a mobile version, which does work on the K700i. And very well so. I will never be lost again, given GPRS reach is near :). Typing the address of the place you're in and waiting maybe 30 seconds for the map to load entirely is enough to let your phone lead the way. Browsing the map is fast enough with GPRS (faster than you can walk, and probably faster than driving a car). It has an easy and simple function to get directions from A to B. So my advice is this: get a nice data bundle with your mobile subscription, browse to http://www.google.com/gmm/ with your phone and download it.
The only thing that worries me is that not only does Google know every keyword I search for, every site I visit and the passwords I use on them, it now knows all places I visit as well. Maybe I should install a camera in my home and broadcast it 24/7 on Google Video, to take away all doubt that there is anything left Google doesn't know about me.
So you can imagine how happy I was to find out that Google Maps has a mobile version, which does work on the K700i. And very well so. I will never be lost again, given GPRS reach is near :). Typing the address of the place you're in and waiting maybe 30 seconds for the map to load entirely is enough to let your phone lead the way. Browsing the map is fast enough with GPRS (faster than you can walk, and probably faster than driving a car). It has an easy and simple function to get directions from A to B. So my advice is this: get a nice data bundle with your mobile subscription, browse to http://www.google.com/gmm/ with your phone and download it.
The only thing that worries me is that not only does Google know every keyword I search for, every site I visit and the passwords I use on them, it now knows all places I visit as well. Maybe I should install a camera in my home and broadcast it 24/7 on Google Video, to take away all doubt that there is anything left Google doesn't know about me.
Synchronizing bookmarks revisited
It's been over two years since I blogged about synchronizing your bookmarks in Firefox. Boy, does time fly!
Since then, the bookmarks synchronizer extension wasn't particularly well maintained. About a gazillion different versions popped up, because every new Firefox version disabled the previous one and a new person would be clever and release an updated bookmarks synchronizer. Which was okay, but required some searching everytime a new Firefox version appeared. Another thing I began to dislike about the bookmarks synchronizer was the fact that it took 15-20 seconds to download my bookmarks and update them everytime when I started Firefox. At some point that gets annoying.
As the attentive reader might have noticed the previous paragraph is written in the past tense. Because when I heard about Google Browser Sync I tried it immediately. And it works so well I installed it at all my systems two minutes later. It's a blessing. Not only does it synchronize your bookmarks transparently and without errors, it also synchronizes saved passwords, cookies, browsing history and open tabs. And if you're afraid uncle Google will read your passwords, don't be, because they're all stored encrypted. And I hope they don't have an uber-unlock-key ;).
Another less well known cool tool for synchronizing bookmarks is BookmarkBridge. This tool can synchronize bookmarks between various browsers, including Mozilla browsers, Internet Explorer and Konqueror. It's not a browser extension so synchronizing is a manual action, and BookmarkBridge doesn't keep them up to date. But still it's pretty useful to have those frequently visited URLs at your fingertips in those non-primary browsers. It's open source and platform independent, with installers for Windows and Linux. It's in the debian packages as well so all it required was 'sudo aptitude install bookmarkbridge'.
Since then, the bookmarks synchronizer extension wasn't particularly well maintained. About a gazillion different versions popped up, because every new Firefox version disabled the previous one and a new person would be clever and release an updated bookmarks synchronizer. Which was okay, but required some searching everytime a new Firefox version appeared. Another thing I began to dislike about the bookmarks synchronizer was the fact that it took 15-20 seconds to download my bookmarks and update them everytime when I started Firefox. At some point that gets annoying.
As the attentive reader might have noticed the previous paragraph is written in the past tense. Because when I heard about Google Browser Sync I tried it immediately. And it works so well I installed it at all my systems two minutes later. It's a blessing. Not only does it synchronize your bookmarks transparently and without errors, it also synchronizes saved passwords, cookies, browsing history and open tabs. And if you're afraid uncle Google will read your passwords, don't be, because they're all stored encrypted. And I hope they don't have an uber-unlock-key ;).
Another less well known cool tool for synchronizing bookmarks is BookmarkBridge. This tool can synchronize bookmarks between various browsers, including Mozilla browsers, Internet Explorer and Konqueror. It's not a browser extension so synchronizing is a manual action, and BookmarkBridge doesn't keep them up to date. But still it's pretty useful to have those frequently visited URLs at your fingertips in those non-primary browsers. It's open source and platform independent, with installers for Windows and Linux. It's in the debian packages as well so all it required was 'sudo aptitude install bookmarkbridge'.
2004/07/17
Synchronizing bookmarks
One of my plans for this weekend was to find a good way to synchronize my browser bookmarks on multiple systems. When you use the internet a lot, and you have a different computer at work, you'll see why you would want a solution for that.
After some time of googling and trying results I think I have found it: Bookmarks Synchronizer for Mozilla Firefox. This synchronization extension saves your bookmarks as an XML file to an FTP server. Set the login data once on every system, and every time you open Firefox you get the most recent version of your single set of bookmarks. Saving and loading the bookmarks does take a short while. My XML file turned out to be 150kB. That isn't too much of an issue, however, as you can start browsing before loading the bookmarks is finished.
If you don't have an FTP account at the ready, there is at least one company which still gives away free web space with FTP access: VillagePhotos. Actually their service is only meant for sharing photos, so you have to configure Bookmark Synchronizer to use a .jpg extension for the XML file. Which of course isn't a problem for saving or loading the data. Thanks VillagePhotos! :)
I've been using Firefox as my main browser for quite some time now, with great satisfaction. If you're looking for a way to synchronize your bookmarks/favorites with a different browser, SiteBar might be the thing for you. This puts a sidebar with bookmarks in your browser, which are stored online. Several free public bookmark servers exist. This could be a satisfactory solution for me, if only it was better integrated with the browser. I need keyboard shortcuts, fast cut-and-paste and drag-and-drop of bookmarks. SiteBar provides none of those. The easy and free online storage is good, and it contains some cool features to share some of your bookmarks online. But as long as I can't store my real browser bookmarks in it, or am provided with a user interface that's as good as the native one of my browser, SiteBar isn't for me.
I can't wait to be at work and have my complete bookmark collection available!
Update: VillagePhotos has recently spawned a new service specifically for hosting any file, not just photos: Ripway. Also free and the service is good. Happy synchronizing!
After some time of googling and trying results I think I have found it: Bookmarks Synchronizer for Mozilla Firefox. This synchronization extension saves your bookmarks as an XML file to an FTP server. Set the login data once on every system, and every time you open Firefox you get the most recent version of your single set of bookmarks. Saving and loading the bookmarks does take a short while. My XML file turned out to be 150kB. That isn't too much of an issue, however, as you can start browsing before loading the bookmarks is finished.
If you don't have an FTP account at the ready, there is at least one company which still gives away free web space with FTP access: VillagePhotos. Actually their service is only meant for sharing photos, so you have to configure Bookmark Synchronizer to use a .jpg extension for the XML file. Which of course isn't a problem for saving or loading the data. Thanks VillagePhotos! :)
I've been using Firefox as my main browser for quite some time now, with great satisfaction. If you're looking for a way to synchronize your bookmarks/favorites with a different browser, SiteBar might be the thing for you. This puts a sidebar with bookmarks in your browser, which are stored online. Several free public bookmark servers exist. This could be a satisfactory solution for me, if only it was better integrated with the browser. I need keyboard shortcuts, fast cut-and-paste and drag-and-drop of bookmarks. SiteBar provides none of those. The easy and free online storage is good, and it contains some cool features to share some of your bookmarks online. But as long as I can't store my real browser bookmarks in it, or am provided with a user interface that's as good as the native one of my browser, SiteBar isn't for me.
I can't wait to be at work and have my complete bookmark collection available!
Update: VillagePhotos has recently spawned a new service specifically for hosting any file, not just photos: Ripway. Also free and the service is good. Happy synchronizing!
Clariti
Clariti. That's what this blog is about. To provide clariti in areas that need it. Let's start by providing some clariti on the name of this blog. Why not use the correct english word? Firstly because that was taken as a Blogspot domain. And secondly because this blog is about IT issues, and last i in Clariti nicely emphasizes that the acronym IT is in it.
In this blog I will publish my thoughts on any technical issues I stumble upon, which I think are worth publishing. Too often when I google for a solution to a specific technical problem, I find many resources that just don't quite get it. In such cases I'll provide what I think is a clearer answer on this site.
There won't be any limit in scope to a specific technical area. Anything I find interesting, I will publish. So I think this resource will be most useful as an archive, again found with a search engine, hopefully providing a clear answer to questions people are looking for and thereby saving some hassle.
And what's in it for me? It is said writing clears your thoughts, and in my experience that is very true. So for me the title is especially applicable.
clar·i·ty:
In this blog I will publish my thoughts on any technical issues I stumble upon, which I think are worth publishing. Too often when I google for a solution to a specific technical problem, I find many resources that just don't quite get it. In such cases I'll provide what I think is a clearer answer on this site.
There won't be any limit in scope to a specific technical area. Anything I find interesting, I will publish. So I think this resource will be most useful as an archive, again found with a search engine, hopefully providing a clear answer to questions people are looking for and thereby saving some hassle.
And what's in it for me? It is said writing clears your thoughts, and in my experience that is very true. So for me the title is especially applicable.
clar·i·ty:
- Clearness of appearance: the clarity of the mountain air.
- Clearness of thought or style; lucidity: writes with clarity and perception.
With this blog I hope to establish both.
2004/01/01
About Clariti
Clariti is written by Aron van Ammers.
