|STAT Statistical Data Analysis : Package Home Page

" /> |STAT Statistical Data Analysis : Package Home Page

|STAT Statistical Data Analysis

Free Data Analysis Programs for UNIX and DOS

by Gary Perlman

Home

Last updated: Accesses since 1998-03-17:

The |STAT Handbook
Data Analysis Programs
on UNIX and MSDOS
and on Linux and Mac OS

Gary Perlman

Handbook Chapters
Preface Introduction to the Package Annotated Example Conventions Used in the Package Data Manipulation Programs Data Analysis Programs DM Manual CALC Manual Program Manual Entries

|STAT Home Page

|STAT News
Package Features
Data Manipulation Programs
Data Manipulation Program Highlights
Data Analysis Programs
Obtaining |STAT
- DOS Version of |STAT
Distribution Conditions
Notes

|STAT is a set of about 30 data manipulation and analysis programs developed by Gary Perlman at the University of California, San Diego and at the Wang Institute. The manipulation programs are general utilities that work with other standard programs like sort. The analysis programs compute most widely used statistics. |STAT programs are designed with the philosophy that individual programs should be designed as tools that do one task well and produce output suitable for input via pipes to |STAT and other programs. Interactive use is supported in the command line interpreter/editor while batch files or shell scripts provide a programming language for complex analyses. Typical usage involves a pipeline of transformations of data followed by input to an analysis program, summarized schematically by:

INPUT DATA | TRANSFORM | ANALYSIS | OUTPUT RESULTS

|STAT News

|STAT turns 35! It was an autumn day in 1979 when I skipped a class to implement the histogram scaling algorithm for the first stat program, hist, which was eventually incorporated into desc.
Sun SPARC Executables
Compiled SunOS versions from 1996 are available in a zip or tar archive.
Web Forms for Analysis Programs
Now you can do some quick analysis from your web browser. Each form has a link to create an example run. See the |STAT Web Forms Page.
Documentation on the Web
The original |STAT handbook and program manual entries are available the Web.
- Handbook (single HTML file) or Handbook (single PDF file)
- Program Manual Entries
DOS |STAT available! The compiled DOS version of |STAT is available along with documentation in a WinZip archive.
DOS |STAT turns 20! In 1986, Fred Horan at Cornell and I ported the code to the PC.
UNIX |STAT turns 25! The first |STAT program, desc, was created in late October of 1979; anova was created April 1980.
An early (1982) paper on the main design ideas in |STAT as available at: statmdf.html
Nice online statistics book: HyperStat, by David M. Lane
Another suite of programs using pipes (for digital signal processing and auditory modelling): |WAVE, by John Culling

Package Features

simple input formats (free-format field-oriented)
flexible data manipulation
several simple lineprinter plotting options
data validation (range and type checking)
consistent option conventions with online help
compiles and runs on any UNIX System (V6, V7, 2.8 BSD, 4.x BSD, System V, ANSI, etc.)
runs on MSDOS/PCDOS 2.x, 3.x, 4.x, 5.x with 96K (IBM, AT&T, Epson, and all compatibles)
usually less than a few seconds per analysis
liberal copyright (but can't be distributed for gain)
in use at hundreds of university, industry, government and research sites for over 20 years

Data Manipulation Programs

abut	join data files beside each other
colex	column extraction/formatting
dm	conditional data extraction/transformation
dsort	multiple key data sorting filter
linex	line extraction
maketrix	make matrix format from free-format input
perm	random/numerical/alphabetical permutation
probdist	probability distribution functions
ranksort	convert data to ranks
repeat	repeat strings or lines in files
reverse	reverse lines, columns, or characters
series	generate an additive series of numbers
transpose	transpose matrix format input
validata	verify data file consistency

Data Manipulation Program Highlights

conditional extraction of rows or columns
data sorting based on multiple keys
line permutation (random and sorted)
matrix formation and transposition functions
conversion of data to ranks
additive series generation + 6 distributions: uniform, normal, t, chi-square, F, binomial
input validation for format and data types
text pagination and formatting with headers
ASCII file archiver for file combination and transfer

Data Analysis Programs

anova	multi-factor analysis of variance, plots
calc	interactive algebraic modeling calculator
contab	contingency tables and chi-square
desc	descriptions, histograms, frequency tables
dprime	signal detection d' and beta calculations
features	tabulate features of items
oneway	one-way anova/t-test, error-bar plots
pair	paired data statistics, regression, plots
rankind	independent conditions rank order analysis
rankrel	related conditions rank order analysis
regress	multiple linear regression and correlation
stats	simple summary statistics
ts	time series analysis, plots

Obtaining |STAT

Instructions updated: 2001-05-12

|STAT is no longer available via e-mail or FTP. It is now stored at an unadvertized Web address. The file: stat.tar.Z is compressed, tar archive of the package. To unpack the archive, you need uncompress or zcat, and tar. To ensure that the distribution restrictions on |STAT are maintained, you must request the location of the file from perlman@acm.org with a message containing:

   I AGREE TO ADHERE TO THE CONDITIONS OF USING |STAT.
   I AGREE NOT TO SHARE THE |STAT LOCATION WITH OTHERS.

Please include a subject with |STAT in it or the spam filters may not let your request through.

After obtaining the file you should run the following commands:

   uncompress stat.tar.Z  # to create stat.tar
   tar xvf stat.tar       # to extract the files

You should then have a stat directory that will take up about a megabyte. In the stat/src directory, you should make any necessary changes to the makefile and then run make to build the package. Installation notes are available. In addition to the online manuals, the file handbook.html contains an HTML version of the |STAT handbook, which is also available in PDF format.

DOS Version of |STAT

The compiled DOS version of |STAT (with documentation) is available in a WinZip archive. No permission or password is needed to access the DOS version, but a note informing perlman@acm.org would be appreciated.

Distribution Conditions

CAREFULLY READ THE FOLLOWING CONDITIONS. IF YOU DO NOT FIND THEM ACCEPTABLE, YOU SHOULD NOT USE |STAT.

|STAT IS PROVIDED "AS IS," WITHOUT ANY EXPRESS OR IMPLIED WARRANTY. THE USER ASSUMES ALL RISKS OF USING |STAT. THERE IS NO CLAIM OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. |STAT MAY NOT BE SUITED TO YOUR NEEDS. |STAT MAY NOT RUN ON YOUR PARTICULAR HARDWARE OR SOFTWARE CONFIGURATION. THE AVAILABILITY OF AND PROGRAMS IN |STAT MAY CHANGE WITHOUT NOTICE. NEITHER MANUFACTURER NOR DISTRIBUTOR BEAR RESPONSIBILITY FOR ANY MISHAP OR ECONOMIC LOSS RESULTING THEREFROM OF THE USE OF |STAT EVEN IF THE PROGRAMS PROVE TO BE DEFECTIVE. |STAT IS NOT INTENDED FOR CONSUMER USE.

CASUAL USE BY USERS NOT TRAINED IN STATISTICS, OR BY USERS NOT SUPERVISED BY PERSONS TRAINED IN STATISTICS, MUST BE AVOIDED. USERS MUST BE TRAINED AT THEIR OWN EXPENSE TO LEARN TO USE THE PROGRAMS. DATA ANALYSIS PROGRAMS MAKE MANY ASSUMPTIONS ABOUT DATA, THESE ASSUMPTIONS AFFECT THE VALIDITY OF CONCLUSIONS MADE BASED ON THE PROGRAMS. REFERENCES TO SOME APPROPRIATE STATISTICAL SOURCES ARE MADE IN THE |STAT HANDBOOK AND IN THE MANUAL ENTRIES FOR SPECIFIC PROGRAMS. |STAT PROGRAMS HAVE NOT BEEN VALIDATED FOR LARGE DATASETS, HIGHLY VARIABLE DATA, NOR VERY LARGE NUMBERS.

You may make copies of any tangible forms of |STAT programs, provided that there is no material gain involved, and provided that the information in this notice accompanies every copy. You may not copy printed documentation unless such duplication is for non- profit educational purposes. You may not provide |STAT as an inducement to buy your software or hardware or any products or services. You may distribute copies of |STAT, provided that mass distribution (such as electronic bulletin boards or anonymous ftp) is not used. You may not modify the source code for any purposes other than getting the programs to work on your system. Any costs in compiling or porting |STAT to your system are your's alone, and not any other parties. You may not distribute any modified source code or documentation to users at any sites other than your own.

Notes

UNIX is a trademark of AT&T, Novell, UNIX Systems Labs, or someone else. On 2000-08-08 I was told that UNIX is a registered trademark of The Open Group in the US and other countries. At one point, UNIX was a trademark of AT&T Bell Laboratories.
MSDOS is a trademark of Microsoft.
|STAT is not a product of any company or organization.
|STAT is used at your own risk.
|STAT should not be used for decision making by non-statisticians.
|STAT is not robust for large datasets or for highly variable data.
|STAT is unsupported, but known bugs are removed.

All rights reserved. No part of this handbook may be reproduced or transmitted in any form or by any means without prior written permission from the author. Non-profit organizations may make copies provided such copies are not made for resale.

First Edition: March 1986
Second Edition: September 1986
Third Edition: March 1987
Fourth Edition: December 2001