Building a better search engine.

Feb 15, 2006 • 1:47 pm | comments (1) by twitter | Filed Under Other Search Engines

With the recent popularity of Google and other major search engines, it seems like many people still feel they can break into the market and offer a “better mousetrap.” Opinions vary on this subject, with many people claiming that companies are just trying to jump on the search bandwagon while it’s hot, and others defending the right to be innovative and come up with a good competing product.

A thread over at Cre8asite Forums discusses one particular search engine, Sensis, and its attempts to provide a better online directory. Apparently the parent company Telstra is very well known to Australians, having developed a sort of infamy, based on the article cited in the Cre8asite post. Could the underlying cause for Sensis/Telstra’s problems be the past history related to state ownership, or is it simply a "dog?"

What this thread gets at is very important to those Google-wannabes out there, and some great points are made about basic flaws in this particular system which makes it less likely that a user would return.

A thread similar in topic to this can also be found at SEW, where someone asked about starting a search engine and got some good answers and guidance.

Here is the Cre8asite thread.

Here is the SEW thread.

Previous story: What do Experts Use When Analyzing Keywords?


Doug Pederson

10/08/2006 05:16 pm

Who wants to help build a search engine. I started building this search engine in 1999. It was just a plain text search then. (.txt) The flowchart to do the line wrap and match hi-lights took 3 days. It worked like a charm. (I'll supply the two page chart) Step one complete. Search and display text results. Next was the ability to search text and call up pictures. If a match was found and the following line had the path to a picture, then that picture was displayed. We scanned in and catalogued 5000+ family pictures and had that as a test base. Changes were made so the search could run as a screen saver. Then with a random option, because sequential becomes very very boring fast. Then came video. Being able to start and stop anywhere in a video clip, slow motion and freeze frame are the main features. I've had a mini dvd camcorder for 4 years now. This program makes computers and video fun. This is desktop search. The very early version (1999) of the search has way less functionality but is valuable as a stripped down search. (I'll supply copies of the early and most recent source.) To make this app a net search, wouldn't be too tuff. Instead of a picture it would be a URL that would be loaded. Linux Mac and C versions shouldn't take much of an effort either. 10,000 lines of Visual Basic 5 with lots of comments and deactivated testing lines. To make this one scalable. Wouldn't be that hard as well. I currently run it as a background job. A few thousand copies searching designated parts of the material, should do the trick. All my notes, emails and net clippings since 1999 amount to 75M At 20,000,000 cps it takes just over 3 seconds. (4 year old laptop) I have looked at some of the others. The more I looked the more I loved this desktop search. The thing I like most is that all my searches can be done in full text mode. With search, it is ALWAYS the context one is looking for. I know the name "Joe Smith" when I search. It is his phone# or a person I met with him that I am interested in. Context context context. The other searches give you jibberish from their index files, before you see the real data. Another key feature to any search is notepad capabilities. This one is very very fast and easy. Before computer search, people had receipe books, phone books address books. They kept notes short and to the point. When looking for something they had to read the text. With computer search you can keep much much more detailed notes and store it all, in one file if you like. Keep every bit of printed matter that is worthwhile. It won't amount to all that much. Mine's next to nothing at 75M. No need for indexing desktop search like Google, Yahoo and Microsoft. Having all your photos, music and telephone videos readily available and this software to play random selections can be entertaining. It's quick to add search detail for these files or video segments. It can flash through photos at 10 a second and faster. Sample your videos, music, pictures and text in the same screen saver. If another search is being used. Augment it's capabilities with this one. There will be no conflicts. This app can be relatively SECURE for text. In display mode the text can not be cut or copied. (The extract capability would have to be disabled.) Having the original file encripted and the decripting taking place on the fly adds to security. Any system can be bypassed by taking pictures or video. Unless they have a photographic memory, everyone needs Desktop Search. This program does: Merge, Search and Replace, Selected photo or song to copy, directory list, differences between 2 files and some lesser functions that make searching useful. This search is leading the pack in simplicity, expandability, usefulness and speed. Gets the data up on the screen fast. This software needs to be taken to the next level. -Maybe develop it with a University. -Maybe a medium to large company that wants to develop search. -Maybe develop it using various forums: Java, VB, Unix, Mac, C etc... -Maybe get some venture capital and 2 developers per project, then do it here. -Maybe opensource. -Maybe all of the above. Next version will have: Search for URL's and put the text into the clipboard. That way I would just have to paste that into the browser. I will also change the text search to default to case sensitive. A control element will allow for over-ride. It will be a little quicker when each line doesn't have to be switched to uppercase. This is relatively simple code and does a lot right now. With a nice interface and some fresh ideas, it can go a long way. I can supply: Latest source code, Very early source code, Flowcharts, Documentation and numerous examples. This search is so fast it will make you laugh. cheers and happy searching.

blog comments powered by Disqus