Welcome to the SurfAgent README !


   * Contents

        o What is SurfAgent?
        o How does it work?
        o Why should I use it instead of the original(s)?
        o Download version 0.2.2 of SurfAgent
        o How to install the software
        o How to use the agent component
        o What you can do to help me
        o Contact information
        o Links to related pages


   * What is SurfAgent?

     SurfAgent is a Web Browsing Assistant that helps its users find
     interesting online newspaper articles. It operates as a proxy,
     monitoring the documents you download and, on your request, analyzes
     them to learn your preferences. You can enter a set of "hub" pages
     (e.g., the frontpage of an online newspaper), and SurfAgent will
     monitor them for links that match your profile.

     In addition to the AI part, SurfAgent is fully compliant with
     Junkbuster, the banner-ad and cookie filtering tool brought to you by
     Junkbusters Corp.


   * How does it work?

     SurfAgent's code is a rewritten version of Junkbuster, modified and
     optimized to work using POSIX threads that share global storage within
     a single process, as opposed to using fork() to start a new process for
     each HTTP request. This enables the proxy to maintain internal state,
     such as user profile information and shared cookies.

     Each user can run their own instance of SurfAgent for maximum privacy,
     but friends who trust each other can share the services of a single
     instance of the proxy. To make this possible, SurfAgent uses the basic
     proxy authentication mechanism offered by the HTTP protocol. Each
     download request must carry an authentication field in the header, in
     order to be matched by an entry in the user database (this is handled
     automatically by your web browser, which only asks for the password the
     first time it accesses the proxy). This is not meant to offer security
     per se, but only to keep trusted users from tripping over each other's
     (virtual) toes :)

     You will notice that once you start using the proxy, each Web page will
     have an extra button appended at the bottom, like this:
                                    SurfAgent
     By clicking on this button, you will be able to access your user
     profile, where you can give reinforcement on documents you like by
     categorizing them according to your various domains of interest, and
     where you can access your list of sites to monitor for interesting new
     articles (and the results of monitoring these sites -- links to
     articles the agent has selected for you).


   * Why should I use it instead of the original(s)?

     Many good reasons :) First off, you get to play with a web agent that
     learns what you like and tries to find interesting articles for you.
     Aren't you curious?

     Then, there are the optimizations: SurfAgent has been carefully tuned
     to squeeze the maximum performance out of multithreading. It no longer
     uses expensive fork() calls to generate separate processes for each
     component (e.g., image or frame) of a Web page.

     Multithreading also allows the sharing of cookies. Cookies are all
     blocked unless special permissions are granted in the cookie
     configuration file. In addition to the original options offered by
     Junkbuster (allowing cookies to pass through bi- or unidirectionally),
     SurfAgent optionally allows for cookies to be handled by the proxy.
     That is, cookies that are not used for authenticating sensitive user
     accounts (e.g., on e-shopping sites), but are required by the visited
     site anyway, can be shared between all users of the proxy. One such
     example is the New York Times, who wants to track your every move on
     their site without offering any useful personalization in return. Using
     SurfAgent's SmartCookie service, you and all your friends get to read
     the NY Times in the name of a single user, without having to enable
     cookies on your browsers.

     The version of Junkbuster maintained by Stefan Waldherr offers to
     replace banner ads with transparent 1  1 pixel GIFs. It maintains two
     separate blocklists, one for images, the other one for html documents.
     Also, a newer version offers an experimental feature that lets users
     circumvent the blocking of html documents misclassified as junk.
     SurfAgent offers optimized image blocking and a well-integrated block
     circumvention for mistakenly blocked URLs. A single blocklist is
     required, and the proxy discriminates between text/html documents and
     images based on the "accept:" field in the browser request header. If
     the browser exclusively expects an image, and its URL is on the
     blocklist, the proxy automatically returns a blank 1  1 GIF image. If
     the browser will accept a text/html document, a page is generated
     containing a message as to why the document was blocked, and offers a
     link through which the document can be accessed in spite of being
     blocked.


   * Download version 0.2.2 of SurfAgent

        o Get the source code for Solaris and Linux (glibc2 required) from

      http://www.cs.colostate.edu/~somlo/SurfAgent/SurfAgent-0-2-2-src.tar.gz

        o SurfAgent should be easily ported to any other platform that
          offers multithreading support with read-write locks. I only have
          access to Solaris and Linux, but am interested if you have any
          suggestions or patches...
        o SurfAgent is Free Software, released under the GNU General Public
          License (see the enclosed file GPL.txt)


   * How to install the software

       1. Download the distribution file SurfAgent-0-2-2-src.tar.gz;
       2. Unpack the distribution file:

          ~/ gunzip SurfAgent-0-2-2-src.tar.gz

          ~/ tar xvf SurfAgent-0-2-2-src.tar

          This will create the directory SurfAgent-0-2-0
       3. Change into the distribution directory:

          ~/ cd SurfAgent-0-2-2

          Then change into the src/ directory:

          ~/SurfAgent-0-2-2/ cd src

       4. Use the make command to build the executable:

          ~/SurfAgent-0-2-2/src/ make all

          This will cause the surfagent executable to be built and copied
          into the distribution directory (~/SurfAgent-0-2-0/)
       5. Change back into the distribution directory:

          ~/SurfAgent-0-2-2/src/ cd ..

       6. Inspect and, if you wish, modify the configuration file
          surfagnt.ini
       7. Start the proxy:

          ~/SurfAgent-0-2-2/ ./surfagent surfagnt.ini

       8. Start your Web browser. Configure it to access the Internet
          through SurfAgent (by default, the proxy will listen for
          connections at port 8000). E.g., on Netscape, click 'Edit', then
          'Preferences', then 'Advanced', then 'Proxies'. Select "Manual
          proxy configuration", click on 'View', then under "HTTP Proxy"
          enter the name of the machine on which you started SurfAgent, and
          under the corresponding "Port" box enter 8000 (or whatever port
          you configured the agent to use in surfagnt.ini)
       9. Start surfing. The first time you download something (after
          performing the above step, and each time you kill and restart your
          browser) SurfAgent will ask for a username and password. Enter
          something that distinguishes you from all the other users of the
          agent, so that your profile won't get confused with that of your
          friends. Alternatively, if you want to be the only user, edit the
          file src/profile.c and change the line

          #define MAX_USERS 20

          to something like

          #define MAX_USERS 1

          Recompile the program. This way, after you've entered your
          login-password combo, the agent will refuse to accept any new
          users. Keep in mind that this password mechanism has not been
          introduced for security, but for the convenience of a group of
          trusted users.


   * How to use the agent component

        o Find and click the following button:

                                     -----------
                                     |SurfAgent|
                                     -----------

          at the bottom of any given Web page you have downloaded.
        o You will be able to access and update your user profile, which
          consists of a collection of "example" documents created by
          averaging all the interesting documents you have found on a
          particular topic. You can start a new topic with the document from
          which you clicked the SurfAgent button by choosing a topic name
          and entering it into the "New Category" box, or you can add the
          document to an existing topic (one that you have created earlier).
        o The SiteMonitor will help you maintain a list of interesting "hub"
          pages (pages with many links to potentially good articles).
          Periodically (currently every 4 hours), SurfAgent will compare all
          the articles linked off all your selected hub pages against your
          topics of interest, and retain the ones that are close enough in a
          list of recommended articles for your convenience.
          Please bear in mid that the agent will need to learn your
          preferences, and that you will need to teach it by creating and
          upgrading your user profile. This activity has been made as
          unobtrusive as possible -- most of the time it requires two extra
          clicks when you're finished reading a good article.


   * What you can do to help me

     Each time you like a document and add it to your user profile, a log
     entry is made in a file (LOGFEEDBACK.txt) in the SurfAgent-0-2-2/
     directory. This file is under your total control, and you can examine
     the log entries that have been made (they will contain the scrambled
     user name, the similarity measure of the current document to all the
     existing categories/topics, and whether it was recommended to you by
     the SiteMonitor or not).
     Please update your profile everytime you find something interesting (if
     you have enough patience to finish reading a news article, consider it
     interesting). When the logfile grows large enough (50 or more log
     entries), please consider e-mailing it to me so that I can analyze it,
     write a few papers, graduate and stop being poor :) Thanks much in
     advance !


   * Contact information

     If you have any further questions or comments, please contact me at
     somlo@cs.colostate.edu


   * Links to related pages

        o Junkbusters Corp. (the authors of the original Junkbuster proxy)

                         http://www.junkbusters.com

        o Stefan Waldherr's version

                     http://www.waldherr.org/junkbuster

--------------------------------------------------------------------------------
