Minnesota APS (MAPS) Catalog of the POSS I Archive Data Format Documentation ============================================================================ BRIEF CATALOG DESCRIPTION The Minnesota Automated Plate Scanner (MAPS) Catalog of the first epoch Palomar Sky Survey (POSS I) is derived from our digitized scans of glass copies of the blue (O) and red (E) plates of the original Palomar Observatory Sky Survey for the 632 fields with Galactic latitudes |b| > 20 degrees. The scans are done in threshold densitometry mode, recording all pixels above the scanning threshold. The operation of the Automated Plate Scanner and the scanning procedures are described in detail in Pennington et al. (1993). The data reduction methods employed in producing this final MAPS Catalog are described in detail in Cabanela et al. (2003). The MAPS Catalog of the POSS I contains coordinates, magnitudes, colors, and several other computed image parameters for all of the matched images on the blue and red plates, for 89,234,404 objects (56,391,694 stars and 32,842,710 galaxies) down to fainter than 21st magnitude (in the blue). The stellar and non-stellar images are separated using a neural network image classifier, that has been trained to the plate limit. The image parameter data for the MAPS Catalog totals 12.96 Gigabytes (13,920,665,616 bytes) whereas the photometry and astrometry data occupy an additional 2.21 Megabytes (2,317,273 bytes). PURPOSE OF THIS FILE This file is meant to provide documentation on the format of data included in the DVD-R and online distributions of the Minnesota Automated Plate Scanner (MAPS) Catalog of the POSS I. This file does not contain more than cursory information on the data reduction methods employed to convert scans of the POSS I plates into a digital catalog of their contents. Much more information on the methods used to produce the MAPS Catalog is available in Cabanela et al. (2003) which should be cited in any work using the MAPS Catalog of the POSS I. SECTION 1] IMAGE PARAMETER CATALOG (AKA the "MAPS Catalog") The final archive of image parameter data from the Minnesota Automated Plate Scanner (MAPS) Catalog of the POSS I is split into 632 data files distributed online or on four DVD-Rs. These data files have names in the format P###.dat, where "###" is the Palomar Sky Survey Field number corresponding to the 632 POSS I fields scanned for the MAPS Catalog. Each MAPS Catalog data file is a binary file consisting of multiple 156 byte records, corresponding to 39 four-byte integers. The byte-order is big-endian, so there may be byte-swap issues on little-endian systems (notably Linux on Intel x86 compatible computers). Source code is provided for reading the MAPS Catalog binary data files (in the source_code/ directory on the 1st DVD of this set) which automatically handles the byte-swapping issues with reading big-endian files on little-endian systems. If you want to write your own code for reading the MAPS Catalog data files, you might want to consult this source code first for guidance. UNITS NOTE: The only non-standard unit used in the Catalog is the APS Encoder Resolution Element or ERE. An ERE equals 0.336 microns and corresponds to the positional precision of the encoder readouts on the APS. Given a POSS I plate scale of 67.12 arcseconds/mm this implies an ERE corresponds to an angle of 0.0226 arcseconds on the sky. CAVEATS and ERRATA: "Missing" Fields: The following two POSS I fields lie at |b| > 20 degrees but are not included in the MAPS Catalog. P003 : Severely scratched and mostly redundant due to the close placement of POSS I fields near the North Celestial Pole. P926 : Very close to the Galactic bulge with large numbers of blended images. Positions at the 24H/0H line: The RA positions for one object in P465 and one in P880 are given in the Catalog as 24:00:00.0. Therefore they appear at the end of the RA-sorted image parameter data files instead of at the beginning where they belong. Erratum versions of the image parameter data files for these two fields are included on DVD01 with the RA positions for these two stars corrected to 00:00:00.0. IMAGE PARAMETERS DATAFILE FORMAT: The first 156 byte record in each data file is a header describing the POSS I field. The contents of this header record is listed here: HEADER RECORD FORMAT: 39 big-endian integers (4 bytes/integer) ---------------------------------------------------------------------------- # Field Name Description ---------------------------------------------------------------------------- 0 POSS Palomar Sky Survey Field (P) Number 1 Emulsion Palomar Sky Survey Emulsion Number 2 EpochDay Epoch of Observation (Day) 3 EpochMon Epoch of Observation (Month) 4 EpochYear Epoch of Observation (Year) 5 NOBJ Number of objects in field 6 NSTARS Number of stars in field 7 NGALAX Number of galaxies in field 8 raminE*100 Minimum Field Right Ascension (1950) (seconds) 9 ramaxE*100 Maximum Field Right Ascension (1950) (seconds) 10 decminE*100 Minimum Field Declination (1950) (arcseconds) 11 decmaxE*100 Maximum Field Declination (1950) (arcseconds) # A NOTICE ABOUT FIELD LIMITS: # Note positions for individual objects (in subsequent records) are stored # in seconds/arcseconds*1000 (not 100) format. If you wish to ensure the # POSS I field limits in the header are inclusive of all data, round # values by 0.01 seconds/arcseconds outward from field center. 12 ra_rms*1000 Right Ascension Position Uncertainty (arcseconds) 13 dec_rms*1000 Declination Position Uncertainty (arcseconds) 14 Omaglim*100 O plate magnitude limit (O magnitude) 15 Emaglim*100 E plate magnitude limit (E magnitude) 16 Oseeing*1000 O plate seeing estimated from PSF (arcseconds) 17 Eseeing*1000 E plate seeing estimated from PSF (arcseconds) 18 Omuth*1000 O surface brightness limit estimated from PSF 19 Emuth*1000 E surface brightness limit estimated from PSF 20 Omusky*1000 O sky surface brightness estimated from PSF 21 Emusky*1000 E sky surface brightness estimated from PSF # The previous four surface brightnesses have # units of magnitude/(arcsecond)^2 22 Ocalsrc O plate photometric calibration source # 0 - only Guide Star Catalog # 1 - CCD calibration # 2 - Photoelectric measurements # 3 - both CCD and Photoelectric measurements 23 Ocalmin O plate photometric calibration minimum diameter 24 Ocalmax O plate photometric calibration maximum diameter # These are "areal" diameters (see photometry # description below) in units of EREs. 25 Orms_08_14*10000 O plate photometric calibration RMS (mag 8 - 14) 26 Orms_14_20*10000 O plate photometric calibration RMS (mag 14 - 20) 27 Orms_20_22*10000 O plate photometric calibration RMS (mag 20 - 22) # These last three rms measures are in units of # O magnitudes 28 Ecalsrc E plate photometric calibration source # 0 - only Guide Star Catalog # 1 - CCD calibration # 2 - Photoelectric measurements # 3 - both CCD and Photoelectric measurements 29 Ecalmin E plate photometric calibration minimum diameter 30 Ecalmax E plate photometric calibration maximum diameter # These are "areal" diameters (see photometry # description below) in units of EREs. 31 Erms_08_14*10000 E plate photometric calibration RMS (mag 8 - 14) 32 Erms_14_20*10000 E plate photometric calibration RMS (mag 14 - 20) 33 Erms_20_22*10000 E plate photometric calibration RMS (mag 20 - 22) # These last three rms measures are in units of # E magnitudes 34 Blank fields (-1 in field to allow endian check) 35 Blank fields (-1 in field to allow endian check) 36 Blank fields (-1 in field to allow endian check) 37 Blank fields (-1 in field to allow endian check) 38 Blank fields (-1 in field to allow endian check) ---------------------------------------------------------------------------- Following the 156-byte header are NOBJ 156-byte records corresponding to all the objects identified in this POSS I field. The data format for each of these object records is identical and outlined below: OBJECT RECORD FORMAT: 39 big-endian integers per record (4 bytes/integer) ---------------------------------------------------------------------------- # Field Name Description ---------------------------------------------------------------------------- 0 starnumO O image raster (identification) number [unique] # The IAU approved designation of a MAPS Catalog # object has the format MAPS-PMMM-NNNNNNN where # "MMM" is the field number (the value of the first # integer of the header/file) and "NNNNNNN" is the # O plate raster number (the value of the first # integer of each object's record). 1 ra*1000 Right Ascension (1950) (seconds) 2 dec*1000 Declination (1950) (arcseconds) 3 XsctO X centroid on O plate (EREs) 4 YsctO Y centroid on O plate (EREs) 5 diaO*1000 Major-Axis Diameter on O plate (arcseconds) 6 magiO*1000 integrated (galactic) O magnitude 7 magdO*1000 Diameter-Magnitude (stellar) O magnitude 8 colori*1000 O-E color (computed using integrated magnitudes) 9 colord*1000 O-E color (computed using D-M magnitudes) 10 mean_sbO*1000 Mean Surface Brightness on O plate # (magnitude/[arcsecond]^2) 11 thetaO*1000 position angle on O plate (degrees) [Measured west to north, so this is the "standard" (North to East) position angle minus 90 degrees.] 12 ellO*1000 ellipticity on O plate 13 galnodO*1000 galnode on O plate 14 PsatO*1000 Percent image saturation on O plate 15 TavgO*1000 Average transmittance on O plate 16 TskyO*1000 Sky transmittance on O plate (from background scan) 17 ReffO*1000 Effective (half-light) radius on O plate # (arcseconds) 18 C42O*1000 C(r100/r50) concentration index on O plate 19 C32O*1000 C(r75/r50) concentration index on O plate 20 Mir1O*1000 First Moment of Image on O plate 21 Mir2O*1000 Second Moment of Image on O plate 22 starnumE E image raster (identification) number 23 XsctE X centroid on E plate (EREs) 24 YsctE Y centroid on E plate (EREs) 25 diaE*1000 Major-Axis Diameter on E plate (arcseconds) 26 mean_sbE*1000 Mean Surface Brightness on E plate # (magnitude/[arcsecond]^2) 27 thetaE*1000 position angle on E plate (degrees) [Measured west to north, so this is the "standard" (North to East) position angle minus 90 degrees.] 28 ellE*1000 ellipticity on E plate (moments analysis method) 29 galnodE*1000 galnode on E plate 30 PsatE*1000 Percent image saturation on E plate 31 TavgE*1000 Average transmittance on E plate 32 TskyO*1000 Sky transmittance on E plate (from background scan) 33 ReffE*1000 Effective (half-light) radius on E plate # (arcseconds) 34 C42E*1000 C(r100/r50) concentration index on E plate 35 C32E*1000 C(r100/r75) concentration index on E plate 36 Mir1E*1000 First Moment of Image on E plate 37 Mir2E*1000 Second Moment of Image on E plate 38 flag O and E image analysis quality flags # This flag is computed as the sum of three numbers # 100*Eduplicates + 10*Oflag + Eflag # 'Eduplicates' is the number of times this E image # has been matched to an O plate object if this # nuumber exceeds 1, otherwise it is set to zero. # 'Oflag' and 'Eflag' are flags for bad image data # from the O and E plates respectively and can # take on the following values: # 0 OK (no problems noted) # 1 Moments analysis error # 2 Image classifier error # 3 Preprocessing ell suggests this is a scratch # 4 Clipped image on scanning stripe boundary # 5 There is no background scan coverage of image # 6 Negative sky intensity estimate ---------------------------------------------------------------------------- SECTION 2] PHOTOMETRIC AND ASTROMETRIC CALIBRATION DATA In addition to the actual MAPS Catalog of images parameter data for objects on the Palomar Sky Survey, we have also archived information on photometric and astrometric calibration This data is all stored in subdirectories in the photom_astrom/ directory on the first DVD in this distribution. For detailed information on methods employed to complete the photometric and astrometric calibration of the MAPS Catalog, consult Cabanela et al. (2003). A) PHOTOMETRIC CALIBRATION DATA There are four files of interest with regards to the photometric calibration for any POSS I field in the MAPS Catalog. P###.DM.fit.txt: This file provides the values of the parameters in the fit of the diameter-magnitude relation used to determine the magnitude of a point-like (non-extended) object. The function fit was a variant of the King and Kormendy form of the PSF functions (PASP 83, 199 [1971] and AJ 78, 255 [1973]) and has the form: mag(diam) = SB0 + 2.5 log10[(1+(diam/R0)^2)^BETA + (diam/RM)^GAMMA] where "diam" is the "areal" diameter in EREs. The five parameters fit to the data are SB0, BETA, R0, GAMMA, and RM. This fit is accomplished via a non-linear least squares fit and is done independently for the O and E plates. Results of the fits are stored in the file along with NSTARS: Number of stars used in the fit NFLAG: Number of stars flagged and not used in the fit MINDIA: The diameter (in ERE) of the smallest star used in the fit. MAXDIA: The diameter (in ERE) of the largest star used in the fit. CALSRC: The calibration source. Same format as image parameters data file header "calsrc" for that field. RMS: RMS of the entire fit. RMS_08_14: RMS of fit between 8th and 14th magnitude RMS_14_20: RMS of fit between 14th and 20th magnitude RMS_20_22: RMS of fit between 20th and 22th magnitude (If RMS undefined, it's value is set to "99.9000") NOTE: "diameter" in this case is an "areal" diameter in which the area of the image (in square EREs) is computed and the diameter is then computed (in EREs) by assuming the object is circular. P###.DM.standards.txt: This file lists the O and E plate objects corresponding to photometric standards (either from the Guide Star Catalog or CCD or Photoelectric measurements). Note that not all Guide Star Catalog objects in a given POSS field were used as calibrators, so some of them are listed as having no match. Some of the stars in the CCD and photoelectric sequences were not used for various reasons including mis-identification, no match with an object in the MAPS Catalog or poor data. These are likewise indicated as no match. Also note that sometimes due to additional processing of some of the projects after the photometric calibration, the starnumbers had changed and we were subsequently unable to make a confident match with an identification number in the MAPS for some of the objects even though the standard was used in the calibration. These "unrecovered" standards are listed in the files using a separate naming convention noted below. The columns are fixed width and contain: MAPS: The full MAPS Catalog object name when available. Otherwise, two other kinds of "names" may be provided. 1) A "hyphen" means this standard was never used for photometric calibration. 2) An entry in the form "(Ostarnum)" or "(Estarnum)" means that this was an "unrecovered" standard (described above). Standard: The names of the standards indicate where data was obtained. gsc: Guide Star Catalog cap: Capilla Peak CCD Photometry ctio: Cerro Tololo CCD Photometry kpno: Kitt Peak CCD Photometry maj: Majewski pre-publication CCD Photometry (only on P323) mcd: McDonald CCD Photometry V, Verr, B-V, BVerr, V-R and VRerr: Standards magnitudes and colors and their respective errors. night: The night observed, when known, is listed in DD/MM/YYYY format. Odia, Omag, Oerr, Oresid: O plate object "areal" diameter, O magnitude (computed from B,V,R magnitudes using standard transformations), the O magnitude uncertainty, and the residual of the O magnitude to the diameter-magnitude function fit. Edia, Emag, Eerr, Eresid: Same for the E plate. used?: The last column, lists if the star was used in the final fit of the diameter-magnitude relation on both plates (OE), the O plate only (O), the E plate only (E), or neither (N). P###.DI.fit.txt: This file records the information on the O and E plate fits to a density-to-intensity transformation, used to compute the integrated magnitudes of non-stellar (galactic) images. The density-to-intensity relation is fit by a function developed by Greg Aldering The density-to-intensity function is assumed to have a form: D(I) = D_MAX*(1 - exp[ALPHA*(I^(BETA*(I^GAMMA)))]) motivated by a physical model for response of photographic grains to incoming photons. Note that in addition to the four parameters fit (ALPHA, BETA, GAMMA, and D_MAX), the intensity I here is (I_STAR+I_SKY)/I_REF, so I_SKY is another parameter to be fit. As before, the fit is performed independently on the O and E plates. The value of D_SKY is determined from a low-resolution background scan of the plate. This allows a check of the D-to-I transformation using the fit I_SKY. In addition to the density-to-intensity functional fit, estimates of the reference intensity (I_REF), threshold intensity (I_THRESH), and seeing (SEEING) are recorded. P###.cuts.txt: This file lists any "cuts" performed on the data. This is done to remove very bright stars and their diffraction halos and regions of nebulosity. When the digitized images of diffraction halos and nebulosity are processed they are often classified as non-stellar and if they match on both plates, they can appear as a ring of "galaxies" around the position of a bright star. For that reason they are removed. There are two kinds of cuts: 1) box cuts: This is a cut of a rectangular region of the plate (E and O plate cuts are performed independently). The format of box cut records is a space-delimited set of 10 variables * E/O: First character on the line, designates the plate on which the cut was performed. * The second column identifies this line as a "box" cut. * The next two columns are the X,Y position (in EREs) of one corner of the rectangular regions cut. Sometimes called the bottom right-hand corner, but not always technically the case. * The next two columns are the X,Y position (in EREs) of the other corner of the rectangular regions cut. Sometimes called the top left-hand corner, but not always technically the case. * The final four columns are estimates of the RA and Dec of the corners. These are estimates and have not been rigorously tested. Use them with caution. 2) Circular cuts: This is a cut of a circular region (usually a bright star). The format of box cut records is a space-delimited set of 8 variables * E/O: First character on the line, designates the plate on which the cut was performed. * The second column identifies this line as a "cir" or circular cut. * The next two columns are the X,Y position (in EREs) of the center of the circular cut. * The radius of the cut in EREs. * The next two columns are estimates of the RA and Dec of the center of the circle. Again, these are estimates and have not been rigorously tested. Use them with caution. * The final column is the radius of the circle in arcseconds. B) ASTROMETRIC CALIBRATION DATA There is one file of interest with regards to the astrometric calibration for any POSS I field in the MAPS Catalog. P#.astrom.txt: The following information about the astrometric fit for this field is recorded in this file: ASTROMETRIC_CAT: The astrometric catalog used as a reference frame ZERO_PT_CORR: X and Y zero-point corrections in microns INITIAL_ROTATION: The rotation of the field (in degrees) to match astrometric frame. NUM_CAL_STARS: The number of stars used in the astrometric calibration FIELD_CENTER_PLATE: X,Y position of plate center (in EREs) FIELD_CENTER_SKY: Position of plate center on the sky RA_RMS: The RMS of the Right Ascension fit (arcseconds) DEC_RMS: The RMS of the Declination fit (arcseconds) All measured POSS I plate coordinates are adjusted using a distortion map to account for non-geometric nonlinearities which occur at the plate edges, presumably due to warping induced by stress when the plates are bent to fit the focal plane of the Oschin Schmidt telescope. The coordinates are then refered to the nominal plate center, rotated and flipped to a common orientation. The plate model that maps these modified coordinates, (x,y) to standard coordinates on the celestial sphere (i.e. the focal-length-normalized gnomic projection, (X,Y), of RA and Dec) is given by: X = dX + (X|x)*x + (X|y)*y + (X|xr2)*(r^2) where r^2 = x^2 + y^2 Y = dY + (Y|x)*x + (Y|y)*y + (Y|yr2)*(r^2) dX and dY are given in microns, and represent the difference between the true and nominal plate centers. The remaining terms are dimensionless derivatives. Note that one cannot apply the plate solution coefficients directly to the plate positions in the catalog because the steps of correcting for distortion, recentering, rotating, flipping, and normalization by focal length are all performed internally to our code. REFERENCES Cabanela, J.E., Humphreys, R.M., Aldering, G.S., Larsen, J.A., Odewahn, S.C., and Thurmes, P.M. 2003, PASP, Accepted for Publication in July 2003. Pennington, R. L., Humphreys, R. M., Odewahn, S. C., Zumach, W. and Thurmes, P. M., 1993, PASP, 105, 521. ---------------------------------------------------------------------------- - This documentation last updated by Juan Cabanela on May 26, 2005.