Thursday, August 19, 2004
New HTTP Proxy Search Engine Available - Open Source
Note: If you're looking for a paid proxy service that is inexpensive and reliable then I fully recommend not wasting your time with free proxies but rather using Your Private Proxy as I have an account with them myself and their service is unparalleled.
Today I came across the latest in the line of open source search engines. It's called AnomicHTTPProxy and it's actually an http proxy with an search engine capabilities. Here are some of it's features as taken from their product description. With optional pre-fetching. HTTP 1.1 with GET/HEAD/POST/CONNECT is supported. This is sufficient for nearly all public web pages. HTTP headers are transparently forwarded. HTTPS connections through target port 443 are transparently forwarded, non-443 connections are suppressed to enhance security. Both (HTTP and HTTPS) proxies share the same proxy port, which is by default port 8080. The proxy 'scrapes' the content that it passes and creates an index that can be shared between every AnomicHTTPProxy daemons. You can use the indexing feature for intranet indexing: you instantly have a search service at hand to index all intranet-served web pages. You don't need to set up a separated search service. And the used PLASMA indexing is not a naive quick-hack but an properly engineered and extremely fast algorithm; it is capable of indexing a nearly unlimited number of pages, without slowing down the search process. One of the interesting features is it's p2p capabilities which reminds me of grub. The proxy contains an index-sharing p2p-based algorithm which creates a global distributed search engine. This spawns a world-wide global search index. The current release is a minimum implementation of this concept and shall prove it's functionality.
By Jason Dowdell at 10:56 AM | Comments (1)