NARA/CER Correspondence

CER November 22 2023

You asked about obtaining additional data components of OPSANAL that we received in the NIPS format, specifically TIRSA, VCIIA, BASFA, SEAFA, VNDBA. You also asked about obtaining EBCDIC versions of the HES Gazetteer.  Please see the attached spreadsheet for the list of NIPS files we have for these series and the EBCDIC Gazetteer files. The spreadsheet also includes two scanned documentation files that apply to data not available in the National Archives Catalog at this time.
As the list shows, we have NIPS for TIRSA (Record Group 218); BASFA; VCIIA; and, years 1964 through 1968 for VNDBA. We only have de-NIPS'd versions of SEAFA and the 1963 and 1969 VNDBA data.
The Record Group 218 version of TIRSA may be difficult to work with. This version is described in the Catalog at https://catalog.archives.gov/id/7423721. The file has only been minimally processed and we received very limited agency documentation for the data, though it's possible the documentation available with the Record Group 330 version of TIRSA may be applicable.  We do not know how these records may differ or match the Record Group 330 TIRSA records. Also, while the description indicates the data covers the period of circa October 1967 to June 1972, from my brief look at the file, it seems the records actually only cover January 1972 to May 1972.
We are also aware that the VNDBA NIPS files may be challenging. We are aware some of the files may contain inconsistencies or dirty data that might cause the program to output incomplete sets of data. For example, the 1964 NIPS appears to have long strings of space characters embedded in some records, which cause NIPSTRAN to stop processing the file and output an incomplete set of records. Unfortunately, we have not had the resources to explore the issues further in detail.
Our Vietnam War data consists of the only files we have in the NIPS format.  We are not aware of any other NIPS-formatted files in NARA's holdings.  For reference, our Vietnam War reference report identifies which series include the NIPS versions of the data. A link to our report is  https://www.archives.gov/research/military/vietnam-war/electronic-data-files.  And, per your question, we did not receive a NIPS format of the Herbicide File (https://catalog.archives.gov/id/623176).

CER February 24 2023

Also, we have recently updated the NARA-prepared user note in the scanned technical documentation files for this HES series. In addition to the minor edits to the user note, we added the "Note on 1971-72 Hamlet Evaluation System 1971 (HES71) NIPS file" to the documentation file for HAMLA and HES71 (132.1.DP.pdf). This note provides more details about the issue I encountered when preparing the ASCII-rendered versions of the 1971-1972 NIPS data file (HES.HES71.72.NIPS) and may be helpful when working with that file.

Rural Hamlet Derived File
You are also correct that we do not have the Rural Hamlet Derived File (RUMDA). The file was part of the larger body of Vietnam War era data we received from the Department of Defense in the 1970s, so you might see it mentioned in some of the documentation. However, when NARA tried to read the tapes in 1976, we discovered the tapes were unreadable so we did not retain the file. The obsolete federal records control schedule NC1-330-77-02, "Refugees / Air Summary File / Rural Hamlet Derived File / Questions Hamlet Evaluation System 1970," indicates the file was approved for immediate disposal as of 1976. If of interest, a link to this schedule is on NARA's website is https://www.archives.gov/files/records-mgmt/rcs/schedules/departments/department-of-defense/office-of-the-secretary-of-defense/rg-0330/nc1-330-77-02_sf115.pdf.

Reference question
Generally, for questions on how to reference records in NARA's custody, we refer researchers to NARA's "Citing Records in the National Archives of the United States," information leaflet. A link to this documentation is https://www.archives.gov/files/publications/general-info-leaflets/17-citing-records.pdf. However we understand our administrative records may not not fit neatly into the leaflet's guidance, so it will probably require your best judgment.
The two pages Ms. Goodsell provided you are Tape Administration and Preservation Evaluation System (TAPES) lists for the RG 330 HES data files. So, a reference for a single page might be:
Tape Administration and Preservation Evaluation System (TAPES) list for RG 330 Hamlet Evaluation System (HES) data files, March 12, 1996, p. 224; Electronic Records Division; National Archives Building, College Park, MD.
If it helps, we added the pages to the RG 330 HES accession dossier to more clearly document the missing reel. So you might consider referencing the dossier in this case. For example: Hamlet Evaluation System (HES) Files, 1967 - 1974, Accession Dossier NN3-330-75-141; Electronic Records Division; National Archives Building, College Park, MD.
Feel free to adapt the above suggestions as you see appropriate.

CER September 9 2022

The three PAAS data files you ordered in 2019 are those from the Record Group 330 set. These files are described in the National Archives Catalog at https://catalog.archives.gov/id/29011649. If you are interested in ASCII versions of these files, I discuss some options below.
The PAAS data file associated with the PAAS technical documentation you are requesting currently (Order 1) is from Record Group 472 (Catalog description at https://catalog.archives.gov/id/631425). The XMIS# for this file is 005847. This file, referred to as the PAAS Master File, contains data for the time period of January 1970 through February 1973. It was used by the Military Assistance Command, Vietnam (MACV) and differs from the version used by the Office of the Secretary of Defense (i.e. the Record Group 330 version). The scanned documentation file 323.1DP.pdf, which we provided in your 2019 order, gives a brief overview of the four different PAAS data files we maintain (I've attached an extract of the specific supplemental note for reference).

ASCII versions of Record Group 330 PAAS data
The Record Group 330 set of files includes the PAAS NIPS file (XMIS# 015434), the Question Record File (NARA "de-NIPS’d" version; XMIS# 005179), and the Person Record File (NARA reformatted version; XMIS# 005180). As discussed in the attached supplemental user note, the Question Record File and the Person Record File are NARA reformatted versions of the original NIPS data. So, all three PAAS files in Record Group 330 contain the same information–they just present the records differently.
Since we have shared NIPSTRAN with you, you could run the PAAS NIPS file through the software to generate ASCII versions of the data. As you are aware, NIPSTRAN would create individual output files for each fixed and periodic set.
The Question Record File and the Person Record File are preserved in EBCDIC, with fixed length fields. From our records, it seems we provided you with exact copies of the files as preserved. We can output these files to ASCII with record delimiters for you. However, we would need to charge you at the per file fee for the two files. Please let us know if you would like to order copies of the ASCII version and we will prepare a price quotation for those.
Lastly, our office is planning to make ASCII versions of the Record Group 330 PAAS files available in the Catalog in near the future. We will be preparing the ASCII-rendered versions of the PAAS NIPS file, as well as converted versions of the Question Record and Person Record files. That said, we do not have a timeline as to when we may complete this project. If you might prefer to receive the ASCII versions via this method, we can reach out to you when the files are available.
We hope that this information is helpful. If you have further questions concerning NARA's electronic records holdings, please contact us by email at cer@nara.gov.

CER May 5 2022

e are happy to know you have made progress in your research since you last spoke with us regarding the Vietnam-era electronic records series in custody of the National Archives (NARA). From your email, we believe you are seeking some further information about how these electronic records were created and maintained while in use by the various military agencies, as well as the archival processing and preservation of the records, including information about the NIPSTRAN program.

Information about the data
You indicated you are working to "enrich and annotate datasets such as the HES-files and others with source-critical provincial and contextual information (such as dates of creation and change, information on survey practice and personnel, etc.)". By this, it is possible you are seeking information about how the agency used and/or maintained the data during their active lifecycle.
This kind of information may be present in the technical documentation for the electronic records. However, the degree of documentation we received for a dataset can vary considerably in terms of volume and detail. Our Vietnam War data reference report (see link above) usually includes page counts of the available technical documentation for a series, which may help get an idea of the volume of documentation we received for a particular set of records. Also, if the records are available online, the technical documentation packages usually contain a List of Documentation that provides an overview of the documentation, including any supplemental documentation, we may have received.
For example, since you identified the Hamlet Evaluation System (HES) files, you may already be familiar with the technical documentation available online for the Hamlet Evaluation System (HES) Files, 1967 - 1974 (https://catalog.archives.gov/id/4616225). To use the 1971 HES71 file as an example (https://catalog.archives.gov/id/4658121), the first page of the Technical Documentation file (132.1DP.pdf) contains the List of Scanned Documentation. This document also indicates additional documentation available for the series. In particular, since you seem to be interested in how the data were managed at the time, you may also wish to review the supplemental documentation listed in the "Supplemental User Note 2: Available Supplemental Documentation" (page 31 of the PDF) for possible items of interest.
We have also made some of our supplemental documentation for these Vietnam War data records available online and linked them in our reference report. A link to the "Selected Supplemental Documentation" section is https://www.archives.gov/research/military/vietnam-war/electronic-data-files#toc-selected-supplemental-documentation.s we mentioned previously, there may be other records regarding the creation and maintenance of these data systems in NARA’s textual holdings. If you have not done so already, you may wish to start by reviewing the Essay on the Potential Research Value of the Operations Analysis (OPSANAL) System Files for an idea of the relationships between the electronic and textual records. Then you might want to search the National Archives Catalog (https://catalog.archives.gov/) to see what else might be available. You might want to start by looking at the records of the Reports and Analysis Directorate in the Office of Civil Operations and Rural Development Support in Record Group 472. A listing of some of the textual records from the Reports and Analysis Directorate is available through the National Archives Catalog at: https://catalog.archives.gov/search?q=%22Office%20of%20Civil%20Operations%20and%20Rural%20Development%20Support%22%20%22Reports%20and%20Analysis%20Directorate%22&f.oldScope=(descriptions%20or%20online)&f.locationIds=33&SearchType=advanced. You might also want to look for records created by the Data Processing Division of the Joint Chiefs of Staff in Record Group 218 and the Department of Defense, Office of the Assistant Secretary of Defense: Systems Analysis (or later Program Analysis and Evaluation) in Record Group 330. We suggest you contact the Archives II Reference Branch at archives2reference@nara.gov for further guidance in locating records of interest.

Institutional knowledge about the records
From your request, we assume you are also seeking background information about how the records were handled by our archival staff once they were transferred to NARA, such as the processing and preservation actions involved. As you may be aware, our electronic records relating to the Vietnam War were generally transferred to NARA in the 1970s and 1980s. Any staff who worked with these records at time of their initial processing and preservation are no longer with NARA. We can, however, attempt to answer any specific questions you may have.
You may wish to consider requesting access to the accessioning dossiers for these records. The accessioning dossiers consist of administrative documentation that staff create and compile about the transfer, processing, and preservation of the records. The dossiers may include memos, standard forms, correspondence, and notes added by the processing archivist that document information gathered during their work. We also maintain preservation job folders, which document the preservation of the records in our archival storage systems.
If you are interested in access to the administrative records we may have for these records, you will need to file a Freedom of Information Act (FOIA) with NARA’s FOIA Officer (General Counsel) to request a review of the records to determine what can be released. You can send your request to foia@nara.gov, or request online at FOIAOnline or https://www.foia.gov/. In your request, specify you are seeking the accessioning dossier held by RRE and then specify which Vietnam War data files for which you would like to see the accessioning dossier.
For example, you may be interested in requesting dossiers for accessions NN3-349-78-001 and NN3-349-79-002, which contain records from Record Group 349: Records of U.S. Forces in Southeast Asia. These two dossiers include a lot of paperwork pertaining to NARA's review and appraisal assessments of many of the Vietnam files in 1978 and 1979. A preliminary page count is about 150 pages for NN3-349-78-001 and about 200 pages for NN3-349-79-002.