Index of /pub/dods/DODS-3.2/3.2.1/
Name | Last Modified | Size |
---|---|---|
Parent Directory | ||
binary | 2017-01-26 07:00 | - |
source | 2017-01-26 07:00 | - |
SECURITY | 2008-05-23 23:26 | 12k |
$Id: README,v 1.6.4.4 2001/10/11 19:29:38 edavis Exp $
README
DODS, version 3.2
Table of Contents
-----------------
1. Introduction
2. DODS source distributions and their dependencies
3. Compiling and installing DODS
4. Using DODS
5. Support
A. Source Code Organization
B. About Data Security
1. Introduction
---------------
DODS is a software framework for scientific data networking. The DODS
framework simplifies all aspects of remote data access. Local data can
be made accessible to remote locations regardless of local storage
format by using DODS servers. Existing, familiar data analysis and
visualization applications can be transformed into DODS clients (i.e.,
applications able to access remote DODS served data).
For more information, visit the DODS web site at
http://www.unidata.ucar.edu/packages/dods/
DODS is a freely available open source package.
2. DODS source distributions and their dependencies
---------------------------------------------------
The DODS software has numerous components. The ones you will require
depends on the data you want to serve and the applications you want to
make into DODS clients. Here is a guide on the DODS source
distribution components:
This information is also available at
http://www.unidata.ucar.edu/packages/dods/home/faq/componentsSource.html
DODS-packages-<ver>.tar.gz: Third party packages used by DODS.
- Results: 3rd-party libraries (i.e., libwww, librx, libz, libtcl, libtk)
- Requires: no other DODS software
- Required by: DODS-dap-<ver>.tar.gz and thus by all other DODS
software.
DODS-dap-<ver>.tar.gz: The DODS Data Access Protocol implementation.
- Results: DODS core libraries (i.e., libdap++.a and libdap++-gui.a)
- Requires: DODS-packages-<ver>.tar.gz
- Required by: all DODS software
DODS-asciival-<ver>.tar.gz: Tool for displaying DODS data in an ASCII format
- Results: asciival
- Requires: DODS-dap-<ver>.tar.gz and DODS-packages-<ver>.tar.gz
- Required by: all DODS servers
DODS-www_interface-<ver>.tar.gz: Tool that provides the DODS web interface
- Results: www_int
- Requires: DODS-dap-<ver>.tar.gz and DODS-packages-<ver>.tar.gz
- Required by: all DODS servers
DODS-dsp-dods-<ver>.tar.gz: The DSP server.
- Results: DSP server components (i.e., dsp_dds, dsp_das, dsp_dods)
- Requires: DODS-dap-<ver>.tar.gz and DODS-packages-<ver>.tar.gz
builds but server will be incomplete without
DODS-asciival-<ver>.tar.gz and DODS-www_interface-<ver>.tar.gz
- Required by: none
DODS-ff-dods-<ver>.tar.gz: The FreeFormND server and some FF tools.
- Results: FF server components (i.e., ff_dds, ff_das, ff_dods) and
FF utilities (i.e., bufform, checkvar, checkform, newform)
- Requires: DODS-dap-<ver>.tar.gz and DODS-packages-<ver>.tar.gz
builds but server will be incomplete without
DODS-asciival-<ver>.tar.gz and DODS-www_interface-<ver>.tar.gz
- Required by: none
DODS-hdf-dods-<ver>.tar.gz: The HDF 4 and HDF-EOS server.
- Results: HDf server components (i.e., hdf_dds, hdf_das, hdf_dods)
- Requires: DODS-dap-<ver>.tar.gz and DODS-packages-<ver>.tar.gz
builds but server will be incomplete without
DODS-asciival-<ver>.tar.gz and DODS-www_interface-<ver>.tar.gz
- Required by: none
DODS-jg-dods-<ver>.tar.gz: The JGOFS server, client library, and an
example client.
- Results: JGOFS server components (i.e., jg_dds, jg_das, jg_dods),
JGOFS client library (i.e., libjg-dods.a), and
JGOFS example client (i.e., list)
- Requires: DODS-dap-<ver>.tar.gz and DODS-packages-<ver>.tar.gz
builds but server will be incomplete without
DODS-asciival-<ver>.tar.gz and DODS-www_interface-<ver>.tar.gz
- Required by: none
DODS-nc3-dods-<ver>.tar.gz: The netCDF 3 server, client library, and
an example client.
- Results: netCDF server components (i.e., nc_dds, nc_das, nc_dods),
netCDF client library (i.e., libnc-dods.a), and
netCDF example client (i.e., dncdump)
- Requires: DODS-dap-<ver>.tar.gz and DODS-packages-<ver>.tar.gz
builds but server will be incomplete without
DODS-asciival-<ver>.tar.gz and DODS-www_interface-<ver>.tar.gz
- Required by: DODS-ncview-<ver>.tar.gz
DODS-idl-cmdln-<ver>.tar.gz: The IDL command-line client.
- Results: IDL client components (i.e., idl_dods.so and numerous .pro files)
- Requires: DODS-dap-<ver>.tar.gz and DODS-packages-<ver>.tar.gz
- Required by: none
DODS-ml-cmdln-<ver>.tar.gz: The Matlab command-line client.
- Results: Matlab client components (i.e., loaddods and writeval)
- Requires: DODS-dap-<ver>.tar.gz and DODS-packages-<ver>.tar.gz
- Required by: none
DODS-ncview-<ver>.tar.gz: The ncview graphical client.
- Results: dncview
- Requires: DODS-dap-<ver>.tar.gz, DODS-packages-<ver>.tar.gz, and
DODS-nc3-dods<ver>.tar.gz
- Required by: none
3. Compiling and installing DODS
--------------------------------
See the file INSTALL in this directory for directions on building
and installing DODS software.
4. Using DODS
-------------
The best place to look for guidance on how to use DODS is the DODS
User Guide which is online at
http://www.unidata.ucar.edu/packages/dods/user/guide-html/
Also, several of the DODS source distributions contain USING files
that give information specific to using the software in that
distribution. See the next section for how to get support for DODS
software.
5. Support
----------
Our goal is to support DODS to the best of our abilities. We think the
best way to do this is to encourage community-based support. If DODS
users will try to follow the support guidelines below, we can all
work to help each other:
1) Consult our online documentation
http://www.unidata.ucar.edu/packages/dods/home/docs.html
- DODS User Guide
http://www.unidata.ucar.edu/packages/dods/user/guide-html/
- DODS Quick Start Guide
http://www.unidata.ucar.edu/packages/dods/user/quick-html/
- DODS Server Installation Guide
http://www.unidata.ucar.edu/packages/dods/user/install-html/
2) Consult our Frequently Asked Questions list
http://www.unidata.ucar.edu/packages/dods/home/faq/
3) Consult the searchable DODS support archive
http://www.unidata.ucar.edu/glimpsedocs/ghdods.html
and the searchable DODS email list archive
http://www.unidata.ucar.edu/glimpsedocs/ghdods-list.html
4) Post your question to the dods@unidata.ucar.edu email list for
the DODS user community to answer. Often, a knowledgable DODS
user will answer your question.
OR
Send your question to support@unidata.ucar.edu (Unidata
support). Your question will be addressed by the appropriate
DODS developer. We try to respond to your email within a few
hours of recieving it, either with an answer, a request for
more information, or a note letting you know we are looking
into the issue.
With either of these routes, your email (or the response) will
be archived and searchable on the web at the URL listed above.
Please consider in advance how to make your question
intelligible to an audience unfamiliar with your work. A few
things that may help:
- include information on the operating system that you are
running on (e.g., paste the output of the Unix command
"uname -a" into your message)
- include information on which DODS component (and the version)
you are using
- If you post a question and recieve an answer in private
emails, please post a brief summary of the solution to
dods@unidata.ucar.edu (or support@unidata.ucar.edu) so that
others can see the solution to the question you raised and so
that the email archive contains the solution to your question.
Also, we'd like to express our thanks to the DODS community for all
the helpful suggestions, bug reports, and patches we have
received. With your help, DODS will continue to improve.
A. Source Code Organization
---------------------------
This README file was found in the top-level of the DODS file
hierarchy; this directory contains the following directories:
bin: Empty until some of the DODS software is built.
etc: Each of the servers is initially installed in the etc/
directory. Use the installServers script (also here) to move the
server(s) to your web daemon's cgi-bin directory. This directory
also contains various support files used during the build of the
DODS source code. It also contains templates for the autoconf
configuration and make init files.
include: Header files, both for the DODS libraries.
lib: Once built, the various DODS libraries are put here. These
libraries include both the DODS core software which is common to
all the servers and reimplemented APIs and the two sample
DODS-IZED APIs (netcdf and jgofs).
src: All the DODS source code lives here.
Within src there are the following:
dap-<ver>: An implementation of the DODS Data Access Protocol. This
is used by all the clients and servers.
packages-<ver>: This directory holds various third-party software
packages used by DODS. At the present time only packages used
by the core software are contained in this directory. That is,
you must supply a copy of the HDF library to build the HDF
server, the Matlab libraries to build the Matlab server,
etc. In the future we hope to include with the servers all the
software they use as long as that software is freely available.
tools/asciival-<ver>:
tools/www_interface-<ver>: These programs are used by the server.
DODS servers are made up of a collection of programs most
of which are specific to a particular data storage
technology (e.g., HDF) but these two programs are common to
all the servers.
clients/idl-cmdln-<ver>: An IDL command line client. This client
accepts DODS URLs and loads data into IDL.
clients/ml-cmdln-<ver>: Like the IDL command line client, but for
Matlab. Both of these clients use the DAP directly.
clients/ncview-<ver>: A client made from a popular NetCDF analysis
program relinked with our (re)implementation of netCDF. You
can used our version of the netCDF library to relink any
program which uses the netCDF API. Some examples of other
programs that have been relinked are Ferret (available from
their web site) and GrADS (still in testing).
dsp-dods-<ver>: A server for the U of Miami's DSP image file format.
nc3-dods-<ver>: A server netcdf files. This is a rewrite of our
older netCDF software, it now uses the new netcdf 3.x
library.
jg-dods-<ver>: A server for JGOFS datasets. This server provides a
way to customize the datatype of each variable in the
dataset. JGOFS datasets (which are all on-line) are
typically limited to either String or Floating point
datatypes. Our JGOFS server provides a way to specify each
variable's datatype independently. No change to the dataset
is required. It is easy to configure this server for
data stored in collections of text files.
ff-dods-<ver>: The FreeForm server. Like the JGOFS, this server is
easy to customize for `homegrown' data formats. This server
also has a rich set of date and time functions which
simplifies setting up catalogs of large data collections.
hdf-dods-<ver>: An HDF 4 and HDF-EOS server.
B. About Data Security
----------------------
There are two levels of security which DODS data servers support: domain
restrictions and user restrictions. In conjunction with a World Wide Web
server, access to a DODS server can be limited to a specific group of users
(authenticated by password), specific machine(s) or a group of machines
within a given domain or domains.
NOTE
DODS versions 3.2 and greater software contains significant improvements
in the way password authentication is handled. Older versions of the DODS
clients prompted for the password with each and every interaction between
client and server. Now credentials may be embedded in URLs and are
remembered and reused for the duration of a session.
The security features of DODS servers depend heavily on the underlying WWW
daemon because we felt this was the best way to solve the thorny problem of
ensuring only authorized users accessed data. By using the daemon's
authorization software we are ensuring that the security checks used by DODS
have been tested by many many sites. In addition, WWW daemons already support
a very full set of security features and many system administrators are
comfortable and confidant with them. The tradeoff with using the web daemon's
security system for our servers is that two security settings must be made
for each group of data to be configured and more than one server may be
needed even if provides access to the same type of data.
Because the security features rely almost entirely on the host machine's WWW
server, the steps required to install a secure DODS server will vary
depending on the WWW server used. Thus, before installing a secure DODS
server, check over your WWW server's documentation to make sure it provides
the following security features: access limits to files in the document root
on a per user and/or per machine basis, and the ability to place CGI scripts
within the document root directory. As an alternative to the second
requirement, a server may provide a way to place access limits on a CGI
script not within the document root directory hierarchy.
IMPORTANT
Because security features are used to protect sensitive or otherwise
important information, once set-up they should be tested until you are
comfortable that they work. You should try accessing from at least one
machine that is not allowed to access your data. If you would like, we
will try to access your data, assuming that our machines are among those
not allowed, to help you evaluate your set-up.
Since the security features are provided by a WWW server, it is highly
likely that they are functional and extensively tested. While problems
with these features have shown up in the past (e.g., the Netscape SSL
server bug) they are generally fixed quickly. Thus there is good reason
to assume that your data are safe if you choose to set-up your DODS
server as a secure one. However, *there is a chance* that a defect in the
WWW server software will allow unauthorized people access; how big that
chance is depends on the WWW server software you use and how extensively
its security features are tested. That level of testing is completely
beyond our control.
It is important to distinguish securing a DODS server from securing data. If
data are served using DODS then those data are also also accessible through a
web browser (although it might be hard to figure out the URLs, it is still
possible for the data to be accessed). So the data themselves need to be
stored in directories that have limited access. If all data access will take
place through a DODS server this limitation can exclude all access *except*
the local machine. This is the case because some the DODS server's function
requires being able to read the data through the local host's web server. For
example, if the DODS server cannot read information about the dataset as DODS
objects then it cannot build the INFO document.
It bears repeating: If you're serving sensitive information with DODS, that
information is accessible two ways, one via the DODS server and two through
the WWW server. You need to make sure *both* are protected.
In the past it was possible to install two or more DODS servers on a computer
and assign different protections to each one. However, in practice this has
proven to be very hard for to configure correctly. In many cases where this
feature was used, a secure server was setup up for one group of data while an
open server was set up for another. It was often the case that all the data
were accessible using the open server! Thus, if you need to secure data and
serve it with DODS, it is best to host all the sensitive information on one
machine and put other data on a second machine with an open-access server. If
you *must* run two or more servers from the same physical host, we suggest
that you configure your web server to see two (or more) virtual hosts. This
will provide the needed separation between the groups of data.
USING A SECURE DODS SERVER
Using a secure DODS sever is transparent if the server is configured to allow
access based on hosts or domains. Give the DODS URL to a client; the server
will respond by answering the request if allowed or with an error message if
access is not allowed.
Accessing a server which requires password authentication is a little
different and varies depending on the type of client being used. All DODS
clients support passing the authentication information along with the URL. To
do this add `<username>:<password>@' before the machine name in a URL. For
example, suppose I have a secure server set up on `www.dods.org' and the
user `guest' is allowed access with the password `demo'. A URL for that
server would start out:
http://guest:demo@www.dods.org/...
For example,
http://guest:demo@www.dods.org/secure/nph-dods/sdata/nc/fnoc1.nc.info
will return the info on the data set fnoc1.nc from a secure server. You
cannot access the data without including the username and password `guest'
and `demo'.
Some clients will pop up a dialog box and prompt for the username and
password. Netscape, and some other web browsers, for example, will do
this. Similarly, some DODS clients (e.g., the geturl test client and some
clients that use our NetCDF interface) will also popup a dialog. For DODS
clients, the general rule is that if the client pops up progress information
it will also prompt for authentication credentials.
CONFIGURING A SERVER
In the following I'll use the Apache 1.3.12 server as an example and describe
how to install a server which limits access to a set of users. While this
example is limited to the Apache server, it should be simple to perform the
equivalent steps for any other WWW server that supports the set of required
security features (See ABOUT DATA SECURITY).
I. Create a directory for the server.
To limit access to a dataset to particular machine, begin by creating a
special directory for the server. This maybe either an additional CGI bin
directory or a directory within the web server's document root. In this
example, I chose the latter.
cd /home/httpd/html/
mkdir secure
II. Establish access limitations for that directory.
Establish the access limitations for this directory. For the Apache server,
this is done either by adding lines to the server's httpd.conf file or by
using a per-directory file. Note: The use of per-directory access limit files
is a configurable feature of the Apache server; look in the server's
httpd.conf file for the value of the AccessFileName resource.
I modified Apache's httpd.conf file so that it contains the following:
# Only valid users can use the server in secure. 7/6/2000 jhrg
<Directory /home/httpd/html/secure>
Options ExecCGI Indexes FollowSymLinks
order deny,allow
deny from all
allow from dcz dcz.dods.org
AuthType Basic
AuthUserFile /etc/httpd/conf/htpasswd.users
AuthGroupFile /etc/httpd/conf/htpasswd.groups
AuthName /secure
require valid-user
satisfy any
</Directory>
# Protect the directory used to hold the secure data.
<Directory /home/httpd/html/sdata>
Options Indexes
order deny,allow
deny from all
allow from dcz dcz.dods.org
AuthType Basic
AuthUserFile /etc/httpd/conf/htpasswd.users
AuthGroupFile /etc/httpd/conf/htpasswd.groups
AuthName /sdata
require valid-user
satisfy any
</Directory>
and
ScriptAlias /secure/ "/home/httpd/html/secure/"
The first group of lines establishes the options allowed for the `secure'
directory, including that it can contain CGI programs. The lines following
that establish that only users in the Apache password file can access the
contents of the directory.
The second group of lines secure the data itself from accesses which bypass
the DODS server.
The ScriptAlias line tells Apache that executable files in the directory are
CGIs. You can also do this by renaming the nph-dods script to nph-dods.cgi
and making sure httpd.conf contains the line:
AddHandler cgi-script .cgi
III. Copy the server into the new directory.
Copy the CGI dispatch program and the server filter programs in to the newly
created directory. Use the `installServers' script for this. The script is
available in the etc directory of our source distributions and is also
bundled with our binary distributions. Note that if you're using the
extension `.cgi' to tell Apache that nph-dods is a CGI you must rename
nph-dods to nph-dods.cgi. If you forget to do that then you will get a Not
Found (404) error from the server and debugging information generated by the
DODS server won't appear in Apache's error_log even if it has been turned on.
You are done.
IV. Tips.
Here are some tips on setting up secure servers:
Using the per-directory limit files makes changing limits easier since the
server reads those every time it accesses the directory, while changes made
to the httpd.conf file are not read until the server is restarted or sent
the HUP signal.
Using httpd.conf for your security configuration seems a bit simpler since
all the information is in one place.
Using the installServers script it is easy to set up a suite of DODS servers
and then use symbolic links (make sure to turn FollowSymLinks on in
httpd.conf) to `mount' datasets under those servers. When doing this look for
`loops' in Apache's DocumentRoot that will allow users to circumvent your
security by accessing data using a different path.
If the protections are set up so that it is impossible for the server host to
access the data and/or the DODS server itself, then an infinite loop can
result. This can be frustrating to debug, but if you see that accesses
generate an endless series of entries in the access_log file, it is likely
that is the problem. Make sure that you have `allow from <server host name>'
set for both the directory that holds the DODS server and that holds the
data. Also make sure that the server's name is set to the full name of the
host.
Configuring a secure DODS server can be frustrating if you're testing the
server using a web browser that remembers passwords. You can turn this
feature off in some browers. Also, the geturl tool supplied with DODS can be
useful to test the server since it will not remember passwords between runs.
Proudly Served by LiteSpeed Web Server at www.opendap.org Port 443