Back to top

Server Issues

Questions about issues with your server or server configuration.

How do I configure a secure (i.e., password protected) server?

Submitted by jimg on Wed, 05/11/2011 - 17:17
Answer

There are two levels of security which DODS data servers support: domain restrictions and user restrictions. In conjunction with a World Wide Web server, access to a DODS server can be limited to a specific group of users (authenticated by password), specific machine(s) or a group of machines within a given domain or domains.

NOTES:

  • DAP 3.4 includes support for Digest authentication, which significantly increases the robustness of password access.
  • DODS versions 3.2 and greater software contains significant improvements in the way password authentication is handled. Older versions of the DODS clients prompted for the password with each and every interaction between client and server. Now credentials may be embedded in URLs and are remembered and reused for the duration of a session.

The security features of DODS servers depend heavily on the underlying WWW daemon because we felt this was the best way to solve the thorny problem of ensuring only authorized users accessed data. By using the daemon's authorization software we are ensuring that the security checks used by DODS have been tested by many many sites. In addition, WWW daemons already support a very full set of security features and many system administrators are comfortable and confidant with them. The tradeoff with using the web daemon's security system for our servers is that two security settings must be made for each group of data to be configured and more than one DODS server may be needed even if you're serving only one type of data.

Because the security features rely almost entirely on the host machine's WWW server, the steps required to install a secure DODS server will vary depending on the WWW server used. Thus, before installing a secure DODS server, check over your WWW server's documentation to make sure it provides the following security features: access limits to files in the document root on a per user and/or per machine basis, and the ability to place CGI scripts within the document root directory. As an alternative to the second requirement, a server may provide a way to place access limits on a CGI script not within the document root directory hierarchy.

IMPORTANT

  • Because security features are used to protect sensitive or otherwise important information, once set-up they should be tested until you are comfortable that they work. You should try accessing from at least one machine that is not allowed to access your data. If you would like, we will try to access your data, assuming that our machines are among those not allowed, to help you evaluate your set-up.
  • Since the security features are provided by a WWW server, it is highly likely that they are functional and extensively tested. While problems with these features have shown up in the past (e.g., the Netscape SSL server bug) they are generally fixed quickly. Thus there is good reason to assume that your data are safe if you choose to set-up your DODS server as a secure one. However, there is a chance that a defect in the WWW server software will allow unauthorized people access; how big that chance is depends on the WWW server software you use and how extensively its security features are tested. That level of testing is completely beyond our control.

It is important to distinguish securing a DODS server from securing data. If data are served using DODS then those data are also also accessible through a web browser (although it might be hard to figure out the URLs, it is still possible for the data to be accessed). So the data themselves need to be stored in directories that have limited access. If all data access will take place through a DODS server this limitation can exclude all access except the local machine. This is the case because some the DODS server's function requires being able to read the data through the local host's web server. For example, if the DODS server cannot read information about the dataset as DODS objects then it cannot build the INFO document.

It bears repeating: If you're serving sensitive information with DODS, that information is accessible two ways, one via the DODS server and two through the WWW server. You need to make sure both are protected.

In the past it was possible to install two or more DODS servers on a computer and assign different protections to each one. However, in practice this has proven to be very hard for to configure correctly. In many cases where this feature was used, a secure server was setup up for one group of data while an open server was set up for another. It was often the case that all the data were accessible using the open server! Thus, if you need to secure data and serve it with DODS, it is best to host all the sensitive information on one machine and put other data on a second machine with an open-access server. If you must run two or more servers from the same physical host, we suggest that you configure your web server to see two (or more) virtual hosts. This will provide the needed separation between the groups of data.

Using a Secure DODS Server

Using a secure DODS sever is transparent if the server is configured to allow access based on hosts or domains. Give the DODS URL to a client; the server will respond by answering the request if allowed or with an error message if access is not allowed.

Accessing a server which requires password authentication is a little different and varies depending on the type of client being used. All DODS clients support passing the authentication information along with the URL. To do this add `:@' before the machine name in a URL. For example, suppose I have a secure server set up on `www.dods.org' and the user `guest' is allowed access with the password `demo'. A URL for that server would start out:

http://guest:demo@test.opendap.org/...

For example,

http://guest:demo@test.opendap.org/dap/data/nc/fnoc1.nc.info

will return the info on the data set fnoc1.nc from a secure server. You cannot access the data without including the username and password 'guest' and 'demo'.

Some clients will pop up a dialog box and prompt for the username and password. Netscape, and some other web browsers, for example, will do this. Similarly, some DODS clients may also popup a dialog.

Configuring a Server

In the following I'll use the Apache 1.3.12 server as an example (also tested on Apache 2.0.40, 07/25/03 jhrg) and describe how to install a server which limits access to a set of users. While this example is limited to the Apache server, it should be simple to perform the equivalent steps for any other WWW server that supports the set of required security features (See ABOUT DATA SECURITY).

  1. Create a directory for the server.

    To limit access to a dataset to particular machine, begin by creating a special directory for the server. This maybe either an additional CGI bin directory or a directory within the web server's document root. In this example, I chose the latter.

        cd /home/httpd/html/
        mkdir secure
  2. Establish access limitations for that directory.

    Establish the access limitations for this directory. For the Apache server, this is done either by adding lines to the server's httpd.conf file or by using a per-directory file. Note: The use of per-directory access limit files is a configurable feature of the Apache server; look in the server's httpd.conf file for the value of the AccessFileName resource.

    I modified Apache's httpd.conf file so that it contains the following:

        # Only valid users can use the server in secure. 7/6/2000 jhrg
    
    	Options ExecCGI Indexes FollowSymLinks
    
    	Order deny,allow
    	Deny from all
    	# ALLOW SERVER (IP OF SERVER) MACHINE TO REQUEST DATA ITSELF
    	Allow from __YOUR_SERVER_HERE__ 
    	Require valid-user
    	# ALL VISITORS NEED USERNAME AND PASS BUT NOT SERVER
    	Satisfy any
    
    	AuthType Basic 
    	AuthUserFile /etc/httpd/conf/htpasswd.users 
    	AuthGroupFile /etc/httpd/conf/htpasswd.groups
    	AuthName "Secure server"
    
    
        # Protect the directory used to hold the secure data.
    
    	Options Indexes
    
    	Order deny,allow
    	Deny from all
    	# ALLOW SERVER (IP OF SERVER) MACHINE TO REQUEST DATA ITSELF
    	Allow from __YOUR_SERVER_HERE__ 
    	Require valid-user
    	# ALL VISITORS NEED USERNAME AND PASS BUT NOT SERVER
    	Satisfy any
    
    	AuthType Basic 
    	AuthUserFile /etc/httpd/conf/htpasswd.users 
    	AuthGroupFile /etc/httpd/conf/htpasswd.groups
    	AuthName "Secure data"

    and

        ScriptAlias /secure/ "/home/httpd/html/secure/"

    The first group of lines establishes the options allowed for the 'secure' directory, including that it can contain CGI programs. The lines following that establish that only users in the Apache password file can access the contents of the directory, with the exception that this server is allowed to access the directory without authentication. This last bit is important because DODS servers sometimes make requests to themselves (e.g., when generating an ASCII response) but don't pass on the authentication information.*

    Regarding the 'Satisfy any' directive, Brock Murch says:

    I thought that one needed an "Allow from all" since I want my users to connect from anywhere, which would have necessitated a "satisfy all" since I needed the passwd authentication as well. I didn't know that the "Deny from all" would still allow anyone in so long as the AuthType etc was included and authentication took place. Since this is the case a "satisfy any" will do as I have denied all ip access except for the server itself. The second group of lines secure the data itself from accesses which bypass the DODS server.

    The ScriptAlias line tells Apache that executable files in the directory are CGIs. You can also do this by renaming the nph-dods script to nph-dods.cgi and making sure httpd.conf contains the line:

        AddHandler cgi-script .cgi

    The AuthType directive selects the type of authentication used. Apache 2.0 supports 'Basic' and 'Digest' while other servers may also support GSS-Negotiate and NTLM. Version 3.4 of the DAP software supports all these authentication schemes, although only Basic and Digest have been thoroughly tested. Configuration of Apache 2.0 for Digest authentication is slightly different then for Basic authentication, but is explained well in Apache's on line documentation.

  3. Copy the server into the new directory.

    Copy the CGI dispatch program and the server filter programs in to the newly created directory. Use the `installServers' script for this. The script is available in the etc directory of our source distributions and is also bundled with our binary distributions. Note that if you're using the extension `.cgi' to tell Apache that nph-dods is a CGI you must rename nph-dods to nph-dods.cgi. If you forget to do that then you will get a Not Found (404) error from the server and debugging information generated by the DODS server won't appear in Apache's error_log even if it has been turned on.

    You are done.

  4. Tips.

    Here are some tips on setting up secure servers:

    Using the per-directory limit files makes changing limits easier since the server reads those every time it accesses the directory, while changes made to the httpd.conf file are not read until the server is restarted or sent the HUP signal.

    Using httpd.conf for your security configuration seems more straightforward since all the information is in one place.

    Using the installServers script it is easy to set up a suite of DODS servers and then use symbolic links (make sure to turn FollowSymLinks on in httpd.conf) to `mount' datasets under those servers. When doing this look for `loops' in Apache's DocumentRoot that will allow users to circumvent your security by accessing data using a different path.

    If the protections are set up so that it is impossible for the server host to access the data and/or the DODS server itself, then an infinite loop can result. This can be frustrating to debug, but if you see that accesses generate an endless series of entries in the access_log file, it is likely that is the problem. Make sure that you have `allow from ' set for both the directory that holds the DODS server and that holds the data. Also make sure that the server's name is set to the full name of the host.

    Configuring a secure DODS server can be frustrating if you're testing the server using a web browser that remembers passwords. You can turn this feature off in some browers. Also, the geturl tool supplied with DODS can be useful to test the server since it will not remember passwords between runs.

  5. Thanks

    * Brock Murch worked out some thorny configuration details for securing the Apache/DODS combination.

How do I find the version of the server I'm talking to?

Submitted by jimg on Wed, 05/11/2011 - 17:15
Answer

Append `.ver' to a DODS URL. Or you can use `version' as the dataset name to get the version of a server without knowing any of the data file names.

returns an XML document that describes the server's software components and their versions in addition to the DAP protocol versions supported by the server. Older servers return a plain text document with less information.

Changes in "deflate" in 3.4 preventing data read.

Submitted by jimg on Wed, 05/11/2011 - 17:11
Answer

What is the problem?

Using geturl, a 3.2 client can read compressed responses from both 3.2 and 3.4.

Using geturl, a 3.4 client can read compressed responses from 3.4 but not 3.2.

Uncompressed responses work without any problems.

Recommendation

The problem is that 3.2 servers return garbled compressed responses. The 3.2 clients, due to a separate problem, don't ever ask a server for a compressed response, so the problem with the responses didn't show up.

When a 3.2 server sends a compressed response, the compressed body winds up preceding the response headers. This results in a response that is too garbled for libcurl to understand.

The fix for the 3.2 servers (introduced in 3.4.0) is to call flush() after the headers, but before the body data, are written.

Why the problem exists: I think this problem shows up because compression is handled by having the body data filtered through a sub-process. The headers are written to a FILE* and then the output of the sub-process is tied to that same FILE*. However, it may be that by using dup2() what is actually happening is that two FILE* are accessing oe underlying file descriptor. By using a flush() call we can synchronize the two writes; without it the xdr calls seem to write data that 'wins the race' (maybe because they are calling flush()?).

What to do about reading from the 3.2 servers?

1) Compression is off by default; warn users about the problem. Tell them how to ask a server its version.

2) Encourage people to upgrade their servers. Explain that a bug will prevent optimal use.

Dataset URLs stopped working in 3.2

Submitted by jimg on Wed, 05/11/2011 - 17:09
Answer

What is the problem?

As of DODS (C++) version 3.2, the DODS servers are accessed through "nph-dods" instead of "nph-nc", "nph-hdf", etc. (For more details on why this change was made, see below.) This means that, once a pre-3.2 server is upgraded to 3.2, all the old DODS URLs no longer work. That is, unless there is a fix put in place.

Some Possible Fixes

  1. copy nph-dods to old locations, e.g., 'cp nph-dods nph-nc'
  2. symbolic links, e.g., 'ln -s ./nph-dods nph-nc'
  3. if using apache server, add some ScriptAlias directives to the configuration of web server. E.g., currently, you probably have a line in your configuration file like: 'ScriptAlias /cgi-bin/ /usr/local/apache/cgi-bin/'. Add another line something like 'ScriptAlias /cgi-bin/dods/nph-nc /usr/local/apache/cgi-bin/dods/nph-dods'

Support in the install scripts (i.e., 'installServers')

The 'installServers' script will make copies of nph-dods in the old locations if the installer desires. It isn't pretty but it will do for a temporary solution.

Once version 3.3 comes out, the install scripts will stop supporting this behavior. Copying nph-dods (or any of the solutions mentioned above) will still work; it is only the install scripts that will stop supporting these patches.

Recommendation

We recommend that all server sites that are upgrading from pre-3.2 servers use one of the above mentioned patches (while planning on dropping support for the old form of URLs at some point in the future). NOTE this doesn't apply to new servers just starting with DODS 3.2

When DODS 3.3 comes out, its server installation scripts will no longer support these patches. So, we recommend that server sites use 3.3 as the deadline for support patch start publishing the new URLs immediately; notify your users that the old form of URLs will be dropped at some point in the future; We therefore re retiring the old until DODS 3.3 is available; begin alerting your users that the old URLs will not be available in the future; and remove patch once you upgrade to DODS 3.3

More on change to nph-dods

One of the new features in DODS (C++) version 3.2 was to reorganize all the different servers to have a single point of access. Pre-3.2 servers had a separate CGI script (e.g., "nph-nc", "nph-ff", "nph-hdf") for each type of DODS server. If a single site served different types of data, users had to know which datasets corresponded with which server. A mismatch between server type and data type resulted in an error.

Version 3.2 servers present a single entry point, the CGI script "nph-dods" automatically (with the help of a dods.ini file) routes requests to the appropriate server software.

My server doesn't return the ASCII or HTML responses

Submitted by jimg on Wed, 05/11/2011 - 16:57
Answer

The ASCII and HTML responses are generated by accessing the server using the URL (a future version of the server may use a different design; this is true for version 3.4 and may be true for some later versions). If you are serving data behind a firewall which uses NAT for address traslation, the DNS lookup for the host name can fail.

Greg Miller from the USGS tracked down this problem and came up with a solution:

I am behind a firewall that uses NAT translation, so if it was relying on DNS to find the address, it would fail. I checked my host file and discovered that Red Hat maps the server name into the loopback address and not the IP address of the ethernet interface. I corrected the host file, and everything works fine.

Thanks Greg!

When I start it I get a message that the catalog is not valid.

Submitted by jimg on Wed, 05/11/2011 - 16:55
Answer

When I start my Aggregation Server, I get the following:



DEBUG: AggServer: CatalogServlet copy 
/usr/local/jakarta-tomcat/webapps/thredds/initialContent/dodsC/ to 
/usr/local/jakarta-tomcat-4.1.27-LE-jdk14/content/thredds/dodsC/ (04-04-19
10:57:08 )
DEBUG: AggServer: catalog config 
</usr/local/jakarta-tomcat-4.1.27-LE-jdk14/content/thredds/dodsC/catalogConfig.xml>
is not valid (04-04-19 10:57:09 )

From: Tony Jolibois <tjolibois at cls.fr>

The error didn't come from the catalog itself, but from the network configuration of my computer. In the configuration catalog of the AS server, there are some http URLs:



<!DOCTYPE catalog SYSTEM "http://www.unidata.ucar.edu/projects/THREDDS/xml/AggServerCatalog.dtd"> 
<catalog name="MERCATOR DODS Aggregation Server Catalog" version="0.6" 
      xmlns="http://www.unidata.ucar.edu/thredds" 
      xmlns:xlink="http://www.w3.org/1999/xlink"> 

My environment was this: I have a firewall, and my computer was not open to Internet, so it could not connect to the two sites http://www.unidata.ucar.edu and http://www.w3.org. I tested the local copy of AggServerCatalog.dtd and InvCatalog.0.6.dtd but it didn't work.

After opening the connection to these two URLs at the firewall, all works fine now.

Conclusion: if your computer cannot connect to these sites, you won't be able to run an Aggregating server.

Thanks Tony for tracking this down and providing this FAQ!

What is the status of the Matlab OPeNDAP Server?

Submitted by jimg on Wed, 05/11/2011 - 16:54
Answer

As of September 2003

The Matlab OPeNDAP Server was written when Matlab 4 was current. The server supports all of the data types Matlab supported at the time. The server was never updated to handle the newer data types (structures and cell arrays, e.g.) because there was no demand for that capability. So, when the current server encounters variables of the newer types in a file, it chokes.

The level of interest in adding these capabilities is currently unclear. If anyone is actively interested in these capabilities, please let us know at support@unidata.ucar.edu. The best way to move forward with this activity would be to find a champion for the Matlab server who can drive the development effort. We can provide them with as much help as they need, although experience with C++ would be required. If no one comes forward we can add it to our schedule but we're booked pretty tight for the next six months.

If you know of someone who might be interested in working to expand the capabilities of the Matlab server, please have them contact us at support@unidata.ucar.edu.

Why can't I get my FF server to serve my data?

Submitted by jimg on Wed, 05/11/2011 - 16:29
Answer

Here are a few suggestions for troubleshooting your FF server:

1) First, test that your format files are defined properly and that it fits your data files by running chkform on them. You can also check that your data is being interpreted as you desire by running newform. Both chkform and newform are FreeForm tools that come with the DODS FreeForm Server distributions. (They should be located in the bin/ directory of your distribution.)

So that newform can display your data, you will need an 'ASCII_output_data' section in your format description file. Like this:

	ASCII_input_data "test"
	Time 1 10 double 4
	Test 12 33 enote 20
	
	ASCII_output_data "test"
	Time 1 10 double 4
	Test 12 33 enote 16

2) If you are serving ASCII data, pay attention to the whitespace in your data files.

  • FreeForm will gag if whitespace extends beyond the line length determined by the format description. For instance, if the format descriptor fits
    • '34.523 1.45'
    FF will give an error on
    • '34.523 1.45 '
  • There needs to be whitespace filler if the data in a line doesn't cover the entire format. For instance, if the format descriptor fits this data
    • '34.456 234.456'
    FF will die on this data
    • '34.456 234.45'
    but will not die on
    • '34.456 234.45 '
  • The enote type doesn't seem to work quite like that. For instance, if the format fits
    • '34.456 2.34e-2'
    FF will die on
    • 34.456 2.3e-2

Large HDF files and/or compressed files fail

Submitted by jimg on Wed, 05/11/2011 - 16:25
Answer

Your server has one or both of the following (easy to fix) problems. The cache directory is too small and/or your server cannot find gzip. If the cache directory is too small then files will be purged from the cache too soon resulting in a huge performance degradation. If your server cannot find gzip, it won't be able to decompress files before serving them.

When the OPeNDAP software is used to serve compressed files (e.g. files compressed using gzip), the files are first decompressed and then stored in a cache; data served are read from those cached files. The location of the cache directory is /usr/tmp by default. This can be changed by editing nph-dods and changing the value of $cache_dir. The software is set by default to limit the size of this directory to 50 MB. However, if you're serving large files, or are experiencing a large volume of traffic, you should increase this. To do so, edit the value of of the second parameter to 'purge_cache()' on line 125 of nph-dods. The cache size is given in MB, so changing the 50 to 100 would increase the cache size from 50MB to 100MB. Finally, the decompression software uses the gzip program to do its work. If your computer does not have the gzip program in its /bin directory you'll need to edit the DODS_Cache.pm so that the correct version of gzip is used. Look in that file for "/bin/gzip" and replace that text with the correct pathname. To figure out where gzip is on you computer, type 'which gzip' in a shell.

How do I report the monthly usage statistics?

Submitted by jimg on Wed, 05/11/2011 - 16:23
Answer

You are under no obligation to report your OPeNDAP server usage statistics to the OPeNDAP project. However, if you are willing to share this information, it will help us direct future software development efforts more effectively.

For the OPeNDAP C++ servers, you can enable the sharing of this information when you install your servers with the installServers script. When the install script asks about gathering access statistics, answer yes ("y") and provide the additional information requested. Your servers usage statistics will only be accessible by the local host (your machine) and the official OPeNDAP machine.