Welcome to the SurfAgent Proxy Home Page!

 

• NEW! JunkBusterMT

  JunkBusterMT is fully compatible with Waldherr's 2.0.2-8 release, but has been mostly rewritten to use POSIX threads. Compiles on Linux, Solaris, and Windows (mingw32). Get the sourcecode here.

   

• Contents

  • What is SurfAgent?
  • How does it work?
  • Why should I use it instead of the original(s)?
  • Download version 0.2.2 of SurfAgent
  • How to install the software
  • How to use the agent component
  • Contact information
  • Links to related pages

   

• What is SurfAgent?

  SurfAgent is a Web Browsing Assistant that helps its users find interesting online newspaper articles. It operates as a proxy, monitoring the documents you download and, on your request, analyzes them to learn your preferences. You can enter a set of "hub" pages (e.g., the frontpage of an online newspaper), and SurfAgent will monitor them for links that match your profile.

  In addition to the AI part, SurfAgent is fully compliant with Junkbuster, the banner-ad and cookie filtering tool brought to you by Junkbusters Corp.

   

• How does it work?

  SurfAgent's code is a rewritten version of Junkbuster, modified and optimized to work using POSIX threads that share global storage within a single process, as opposed to using fork() to start a new process for each HTTP request. This enables the proxy to maintain internal state, such as user profile information and shared cookies; also, various internal lists such as the blocklist and cookie list can be accessed more efficiently.

  Each user can run their own instance of SurfAgent for maximum privacy, but friends who trust each other can share the services of a single instance of the proxy. To make this possible, SurfAgent uses the basic proxy authentication mechanism offered by the HTTP protocol. Each download request must carry an authentication field in the header, in order to be matched by an entry in the user database (this is handled automatically by your web browser, which only asks for the password the first time it accesses the proxy). This is not meant to offer security per se, but only to keep trusted users from tripping over each other's (virtual) toes :)

  You will notice that once you start using the proxy, each Web page will have an extra button appended at the bottom, like this:

  SurfAgent

  By clicking on this button, you will be able to access your user profile, where you can give reinforcement on documents you like by categorizing them according to your various domains of interest, and where you can access your list of sites to monitor for interesting new articles (and the results of monitoring these sites -- links to articles the agent has selected for you).

   

• Why should I use it instead of the original(s)?

  Many good reasons :) First off, you get to play with a web agent that learns what you like and tries to find interesting articles for you. Aren't you curious?

  Then, there are the optimizations: SurfAgent has been carefully tuned to squeeze the maximum performance out of multithreading. It no longer uses expensive fork() calls to generate separate processes for each component (e.g., image or frame) of a Web page.

  Multithreading also allows the sharing of cookies. Cookies are all blocked unless special permissions are granted in the cookie configuration file. In addition to the original options offered by Junkbuster (allowing cookies to pass through bi- or unidirectionally), SurfAgent optionally allows for cookies to be handled by the proxy. That is, cookies that are not used for authenticating sensitive user accounts (e.g., on e-shopping sites), but are required by the visited site anyway, can be shared between all users of the proxy. One such example is the New York Times, who wants to track your every move on their site without offering any useful personalization in return. Using SurfAgent's SmartCookie service, you and all your friends get to read the NY Times in the name of a single user, without having to enable cookies on your browsers.

  The version of Junkbuster maintained by Stefan Waldherr offers to replace banner ads with transparent 1 × 1 pixel GIFs. It maintains two separate blocklists, one for images, the other one for html documents. Also, a newer version offers an experimental feature that lets users circumvent the blocking of html documents misclassified as junk. SurfAgent offers optimized image blocking and a well-integrated block circumvention for mistakenly blocked URLs. A single blocklist is required, and the proxy discriminates between text/html documents and images based on the "accept:" field in the browser request header. If the browser exclusively expects an image, and its URL is on the blocklist, the proxy automatically returns a blank 1 × 1 GIF image. If the browser will accept a text/html document, a page is generated containing a message as to why the document was blocked, and offers a link through which the document can be accessed in spite of being blocked.

   

• Download version 0.2.2 of SurfAgent

  • Get the source code for Solaris and Linux (glibc2 required).
  • SurfAgent should be easily ported to any other platform that offers multithreading support with read-write locks. I only have access to Solaris and Linux, but am interested if you have any suggestions or patches...
  • SurfAgent is Free Software, released under the GNU General Public License

   

• How to install the software

  1. Download the distribution file SurfAgent-0-2-2-src.tar.gz;
  2. Unpack the distribution file:

     ~/> gunzip SurfAgent-0-2-2-src.tar.gz
     

     ~/> tar xvf SurfAgent-0-2-2-src.tar

     This will create the directory SurfAgent-0-2-2
  3. Change into the distribution directory:

     ~/> cd SurfAgent-0-2-2

     Then change into the src/ directory:

     ~/SurfAgent-0-2-2/> cd src

  4. Use the make command to build the executable:

     ~/SurfAgent-0-2-2/src/> make all

     This will cause the surfagent executable to be built and copied into the distribution directory (~/SurfAgent-0-2-2/)
  5. Change back into the distribution directory:

     ~/SurfAgent-0-2-2/src/> cd ..

  6. Inspect and, if you wish, modify the configuration file surfagnt.ini
  7. Start the proxy:

     ~/SurfAgent-0-2-2/> ./surfagent surfagnt.ini

  8. Start your Web browser. Configure it to access the Internet through SurfAgent (by default, the proxy will listen for connections at port 8000). E.g., on Netscape, click 'Edit', then 'Preferences', then 'Advanced', then 'Proxies'. Select "Manual proxy configuration", click on 'View', then under "HTTP Proxy" enter the name of the machine on which you started SurfAgent, and under the corresponding "Port" box enter 8000 (or whatever port you configured the agent to use in surfagnt.ini)
  9. Start surfing. The first time you download something (after performing the above step, and each time you kill and restart your browser) SurfAgent will ask for a username and password. Enter something that distinguishes you from all the other users of the agent, so that your profile won't get confused with that of your friends. Alternatively, if you want to be the only user, edit the file src/profile.c and change the line

     #define MAX_USERS 20

     to something like

     #define MAX_USERS 1

     Recompile the program. This way, after you've entered your login-password combo, the agent will refuse to accept any new users. Keep in mind that this password mechanism has not been introduced for security, but for the convenience of a group of trusted users.

   

• How to use the agent component

  • Find and click the following button:

    SurfAgent

    at the bottom of any given Web page you have downloaded.
  • You will be able to access and update your user profile, which consists of a collection of "example" documents created by averaging all the interesting documents you have found on a particular topic. You can start a new topic with the document from which you clicked the SurfAgent button by choosing a topic name and entering it into the "New Category" box, or you can add the document to an existing topic (one that you have created earlier).
  • The SiteMonitor will help you maintain a list of interesting "hub" pages (pages with many links to potentially good articles). Periodically (currently every 4 hours), SurfAgent will compare all the articles linked off all your selected hub pages against your topics of interest, and retain the ones that are close enough in a list of recommended articles for your convenience.
    Please bear in mid that the agent will need to learn your preferences, and that you will need to teach it by creating and upgrading your user profile. This activity has been made as unobtrusive as possible -- most of the time it requires two extra clicks when you're finished reading a good article.

   

• Contact information

  If you have any further questions or comments, please contact me at somlo@cs.colostate.edu

   

• Links to related pages

  • Junkbusters Corp. (the authors of the original Junkbuster proxy)
  • Stefan Waldherr's version