[apache] / ROADMAP

APACHE 2.x ROADMAP
==================
Last modified at [$Date: 2002/10/19 17:41:51 $]


INTRODUCTION
------------
The Apache HTTP Server project must balance two competing and disjoint
objectives: maintain stable code for third party authors, distributors and
most importantly users so that bug and security fixes can be quickly adopted
without significant hardship due to user-visible changes; and continue the
development process that requires ongoing redesign to correct earlier
oversights and to add additional features.

The Apache HTTP Server, through version 2.0, used the Module Magic Number (MMN)
to reflect API changes.  This had the shortcoming of often leaving users
hunting to replace binary third party modules that were now incompatible.
This also left module authors searching through the API change histories to
determine the exact cause for the MMN change and whether their module was
affected.

With the simultaneous release of Apache 2.2-stable and Apache 2.3-development,
the Apache HTTP Server project is moving towards a more predictable stable
release cycle, while allowing forward progress to occur without concern
for breaking the stable branch.  This document explains the rationale between
the two versions and their behavior.


STABLE RELEASES, 2.{even}.{revision}
------------------------------------ 

All even numbered releases will be considered stable revisions. 

Stable revisions will retain forward compatiblity to the maximum
possible extent.  Features may be added during minor revisions, and
features may be deprecated by making appropriate notations in the
documentation, but no features may be removed.

In essence, that implies that you can upgrade from one minor revision
to the next with a minimum of trouble.  In particular, this means:

  * The Module API will retain forward compatibility.
    It will not be necessary to update modules to work with new
    revisions of the stable tree.

  * The run-time configuration will be forward compatible.
    No configuration changes will be necessary to work with new
    revisions of the stable tree.

  * Compile-time configuration will be forward compatible.
    The configure command line options that work in one release
    of the stable tree will also work in the next release.

As always, it will be necessary to test any new release to assure
that it works correctly with a particular configuration and a 
particular set of modules, but every effort will be made to assure
that upgrades are as smooth as possible.

In addition, the following development restrictions will aid in 
keeping the stable tree as safe as possible:

  * No 'Experimental' modules; while it may be possible (based on API changes
    required to support a given module) to load a 2.3-development module into
    a 2.2-stable build of Apache, there are no guarantees.  Experimental 
    modules will be introduced to the 2.3-development versions and either
    added to 2.2-stable once they are proven and compatible, or deferred
    to the 2.4-stable release if they cannot be incorporated in the current
    stable release due to API change requirements.

  * The stable CVS tree should not remain unstable at any time.  Atomic commits 
    aught be used to introduce code from the development version to the stable 
    tree.  At any given time a security release may be in preparation, 
    unbeknownst to other contributors.  At any given time, testers may be
    checking out CVS head to confirm that a bug has been corrected.  And as
    all code was well-tested in development prior to committing to the stable
    tree, there is really no reason for this tree to be broken for more than 
    a few minutes during a lengthy commit.

In order to avoid 'skipped' release numbers in the stable releases, the
Release Manager will generally roll a release candidate (APACHE_#_#_#_RC#)
tag.  Release Candidate tarballs will be announced to the
stable-testers@httpd.apache.org for the stable tree.  Then, the participants
will vote on the quality of the proposed release tarball.

The final APACHE_#_#_# tag will not exist until the APACHE_#_#_#_RC# candidate
has passed the usual votes to release that version.  Only then is the final
tarball packaged, removing all -rc# designations from the version number, and
tagging the tree with the release number.

DEVELOPMENT RELEASES, 2.{odd}.{revision}
-----------------------------------------

All odd numbered releases designate the 'next' possible stable release,
therefore the current development version will always be one greater than
the current stable release.  Work proceeds on development releases, permitting
the modification of the MMN at any time in order to correct deficiencies 
or shortcomings in the API.  This means that modules from one development
release to another may not be binary compatible, or may not successfully
compile without modification to accomodate the API changes.

The only 'supported' development release at any time will be the most
recently released version.  Developers will not be answering bug reports
of older development releases once a new release is available.  It becomes
the resposibility of the reporter to use the latest development version
to confirm that any issue still exists.

Any new code, new API features or new ('experimental') modules may be
promoted at any time to the next stable release, by a vote of the project
contributors.  This vote is based on the technical stability of the new
code and the stability of the interface.  Once moved to stable, that feature
cannot change for the remainder of that stable release cycle, so the vote must
reflect that the final decisions on the behavior and naming of that new
feature were reached.  Vetos continue to apply to this choice of introducing
the new work to the stable version.

At any given time, when the quality of changes to the development branch
is considered release quality, that version may become a candidate for the
next stable release.  This includes some or all of the API changes, promoting
experimental modules to stable or deprecating and eliminating older modules
from the last stable release.  All of these choices are considered by the
project as a group in the interests of promoting the stable release, so that
any given change may be 'deferred' for a future release by the group, rather 
than introduce unacceptable risks to adopting the next stable release.

Third party module authors are strongly encouraged to test with the latest
development version.  This assures that the module will be ready for the next
stable release, but more importantly, the author can react to shortcomings
in the API early enough to warn the dev@httpd.apache.org community of the
shortcomings so that they can be addressed before the stable release.  The
entire burden is on the module author to anticipate the needs of their module
before the stable release is created.  Once a new stable release cycle has
begun, that API will be present for the lifetime of the stable release.  Any
desired changes in the stable versions must wait for inclusion into the next
release cycle.

When deciding to promote a development tree to being stable, a determination
should be made whether the changes since the last stable version warrant a
major version bump.  That is, if 2.2 is the current stable version and 2.3 is
'ready' to become stable, the group needs to decide if the next stable
version is 2.4 or 3.0.  One suggested rule of thumb is that if it requires
too much effort to port a module from 2.2 to 2.4, then the stable version
should be labeled 3.0.

In order to ease the burden of creating development releases, the process
for packaging a development releases is less formal than for the stable
release.  This strategy reflects the fact that while in development, versions
are cheap.  Development releases may be classified as alpha, beta, or GA
to reflect the group's perceived stability of the tree.  Development releases
may be made at any time by any committer.

Please read the following link for a more detailed description of the
development release strategy:

http://httpd.apache.org/dev/release.html


WORKS IN PROGRESS
-----------------

    * Source code should follow style guidelines.
      OK, we all agree pretty code is good.  Probably best to clean this
      up by hand immediately upon branching a 2.1 tree.
      Status: Justin volunteers to hand-edit the entire source tree ;)

      Justin says:
        Recall when the release plan for 2.0 was written:
            Absolute Enforcement of an "Apache Style" for code.
        Watch this slip into 3.0.

      David says:
        The style guide needs to be reviewed before this can be done.
        http://httpd.apache.org/dev/styleguide.html
        The current file is dated April 20th 1998!

        OtherBill offers:
          It's survived since '98 because it's welldone :-)  Suggest we
          simply follow whatever is documented in styleguide.html as we
          branch the next tree.  Really sort of straightforward, if you
          dislike a bit within that doc, bring it up on the dev@httpd
          list prior to the next branch.

      So Bill sums up ... let's get the code cleaned up in CVS head.
      Remember, it just takes cvs diff -b (that is, --ignore-space-change)
      to see the code changes and ignore that cruft.  Get editing Justin :)

    * revamp the input filter syntax to provide for ordering of
      filters created with the Set{Input|Output}Filter and the
      Add{Input|Output}Filter directives.  A 'relative to filterx' 
      syntax is definately preferable.

    * Platforms that do not support fork (primarily Win32 and AS/400)
      Architect start-up code that avoids initializing all the modules 
      in the parent process on platforms that do not support fork.

    . Better yet - not only inform the startup of which phase it's in,
      but allow the parent 'process' to initialize shared memory, etc,
      and create a module-by-module stream to pass to the child, so the
      parent can actually arbitrate the important stuff.

    * Replace stat [deferred open] with open/fstat in directory_walk.
      Justin, Ian, OtherBill all interested in this.  Implies setting up
      the apr_file_t member in request_rec, and having all modules use
      that file, and allow the cleanup to close it [if it isn't a shared,
      cached file handle.]

    * The Async Apache Server implemented in terms of APR.
      [Bill Stoddard's pet project.]
      Message-ID: <008301c17d42$9b446970$01000100@sashimi> (dev@apr)

        OtherBill notes that this can proceed in two parts...

           Async accept, setup, and tear-down of the request 
           e.g. dealing with the incoming request headers, prior to
           dispatching the request to a thread for processing.
           This doesn't need to wait for a 2.x/3.0 bump.

           Async delegation of the entire request processing chain
           Too many handlers use stack storage and presume it is 
           available for the life of the request, so a complete 
           async implementation would need to happen 3.0 release.

    * Add a string "class" that combines a char* with a length
      and a reference count.  This will help reduce the number
      of strlen and strdup operations during request processing.
      Including both the length and allocation will save us a ton 
      of reallocation we do today, in terms of string manipulation.

        OtherBill asks if this is really an APR issue, not an HTTPD issue?


MAKING APACHE REPOSITORY-AGNOSTIC
(or: remove knowledge of the filesystem)

[ 2002/10/01: discussion in progress on items below; this isn't
  planned yet ]

    * dav_resource concept for an HTTP resource ("ap_resource")

    * r->filename, r->canonical_filename, r->finfo need to
      disappear. All users need to use new APIs on the ap_resource
      object.
      
      (backwards compat: today, when this occurs with mod_dav and a
       custom backend, the above items refer to the topmost directory
       mapped by a location; e.g. docroot)

      Need to preserve a 'filename'-like string for mime-by-name
      sorts of operations.  But this only needs to be the name itself
      and not a full path.

      Justin: Can we leverage the path info, or do we not trust the
              user?

      gstein: well, it isn't the "path info", but the actual URI of
              the resource. And of course we trust the user... that is
              the resource they requested.
              
              dav_resource->uri is the field you want. path_info might
              still exist, but that portion might be related to the
              CGI concept of "path translated" or some other further
              resolution.
              
              To continue, I would suggest that "path translated" and
              having *any* path info is Badness. It means that you did
              not fully resolve a resource for the given URI. The
              "abs_path" in a URI identifies a resource, and that
              should get fully resolved. None of this "resolve to
              <here> and then we have a magical second resolution
              (inside the CGI script)" or somesuch.
   
      Justin: Well, let's consider mod_mbox for a second.  It is sort of
              a virtual filesystem in its own right - as it introduces
              it's own notion of a URI space, but it is intrinsically
              tied to the filesystem to do the lookups.  But, for the
              portion that isn't resolved on the file system, it has
              its own addressing scheme.  Do we need the ability to
              layer resolution?

    * The translate_name hook goes away

      Wrowe altogether disagrees.  translate_name today even operates
      on URIs ... this mechansim needs to be preserved.
    
    * The doc for map_to_storage is totally opaque to me. It has
      something to do with filesystems, but it also talks about
      security and per_dir_config and other stuff. I presume something
      needs to happen there -- at least better doc.

      Wrowe agrees and will write it up.

    * The directory_walk concept disappears. All configuration is
      tagged to Locations. The "mod_filesystem" module might have some
      internal concept of the same config appearing in multiple
      places, but that is handled internally rather than by Apache
      core.

      Wrowe suggests this is wrong, instead it's private to filesystem
      requests, and is already invoked from map_to_storage, not the core
      handler.  <Directory > and <Files > blocks are preserved as-is,
      but <Directory > sections become specific to the filesystem handler
      alone.  Because alternate filesystem schemes could be loaded, this
      should be exposed, from the core, for other file-based stores to 
      share. Consider an archive store where the layers become 
      <Directory path> -> <Archive store> -> <File name>
   
      Justin: How do we map Directory entries to Locations?
 
    * The "Location tree" is an in-memory representation of the URL
      namespace. Nodes of the tree have configuration specific to that
      location in the namespace.
      
      Something like:
      
      typedef struct {
          const char *name;  /* name of this node relative to parent */

          struct ap_conf_vector_t *locn_config;

          apr_hash_t *children; /* NULL if no child configs */
      } ap_locn_node;

      The following config:
      
      <Location /server-status>
          SetHandler server-status
          Order deny,allow
          Deny from all
          Allow from 127.0.0.1
      </Location>
      
      Creates a node with name=="server_status", and the node is a
      child of the "/" node. (hmm. node->name is redundant with the
      hash key; maybe drop node->name)
      
      In the config vector, mod_access has stored its Order, Deny, and
      Allow configs. mod_core has stored the SetHandler.
      
      During the Location walk, we merge the config vectors normally.
      
      Note that an Alias simply associates a filesystem path (in
      mod_filesystem) with that Location in the tree. Merging
      continues with child locations, but a merge is never done
      through filesystem locations. Config on a specific subdir needs
      to be mapped back into the corresponding point in the Location
      tree for proper merging.

    * Config is parsed into a tree, as we did for the 2.0 timeframe,
      but that tree is just a representation of the config (for
      multiple runs and for in-memory manipulation and usage). It is
      unrelated to the "Location tree".

    * Calls to apr_file_io functions generally need to be replaced
      with operations against the ap_resource. For example, rather
      than calling apr_dir_open/read/close(), a caller uses
      resource->repos->get_children() or somesuch.

      Note that things like mod_dir, mod_autoindex, and mod_negotation
      need to be converted to use these mechanisms so that their
      functions will work on logical repositories rather than just
      filesystems.

    * How do we handle CGI scripts?  Especially when the resource may
      not be backed by a file?  Ideally, we should be able to come up
      with some mechanism to allow CGIs to work in a
      repository-independent manner.

      - Writing the virtual data as a file and then executing it?
      - Can a shell be executed in a streamy manner?  (Portably?)
      - Have an 'execute_resource' hook/func that allows the
        repository to choose its manner - be it exec() or whatever.
        - Won't this approach lead to duplication of code?  Helper fns?

      gstein: PHP, Perl, and Python scripts are nominally executed by
              a filter inserted by mod_php/perl/python. I'd suggest
              that shell/batch scripts are similar.

              But to ask further: what if it is an executable
              *program* rather than just a script? Do we yank that out
              of the repository, drop it onto the filesystem, and run
              it? eeewwwww...
              
              I'll vote -0.9 for CGIs as a filter. Keep 'em handlers.

      Justin: So, do we give up executing CGIs from virtual repositories?
              That seems like a sad tradeoff to make.  I'd like to have
              my CGI scripts under DAV (SVN) control.

    * How do we handle overlaying of Location and Directory entries?
      Right now, we have a problem when /cgi-bin/ is ScriptAlias'd and
      mod_dav has control over /.  Some people believe that /cgi-bin/
      shouldn't be under DAV control, while others do believe it
      should be.  What's the right strategy?