Vietname Data Base (VNDBA)

The Vietname Data Base (VNDBA) was part of the Operation Analysis System (OPSANAL).

The data base contains information on ground combat operations during the Vietnam War. Locations seem to be encoded using Universal Transverse Mercator coordinate system (UTM).

Reading the NIPS files

Based on a comment from NARA/CER Correspondence:

We are also aware that the VNDBA NIPS files may be challenging. We are aware some of the files may contain inconsistencies or dirty data that might cause the program to output incomplete sets of data. For example, the 1964 NIPS appears to have long strings of space characters embedded in some records, which cause NIPSTRAN to stop processing the file and output an incomplete set of records. Unfortunately, we have not had the resources to explore the issues further in detail.

RG330.VNDBA.N64

PyNIPS can read 336 records from the file:

We can not reproduce the issue described by NARA that NIPSTRAN stops processing (using NIPSTRAN Version 3.5.2).

RG330.VNDBA.N65

PyNIPS can read 338 records from the file:

RG330.VNDBA.NIP66

PyNIPS can read 127432 records:

RG330.VNDBA.NIP67

PyNIPS can read 144008 records:

RG330.VNDB.NIPA68, RG330.VNDB.NIPB68, RG330.VNDB.NIPC68

PyNIPS can read 86589 records from RG330.VNDB.NIPA68:

However, none can be read from RG330.VNDB.NIPB68 or RG330.VNDB.NIPB68 directly by invoking the nipsext info command. It seems like the the two files do not contain a File Format Table (FFT) (determined by inspecting logical records with the nipsext logical command).

We can re-use the FFT from the initial file with following Python script:

import nips.logical_record
from nips.fft import DataFileControlRecord, FileFormatTable
from nips.data_file_record import DataFileRecord


def main(nips_files):

    fft = FileFormatTable()

    record_count = 0

    for nips_file_path in nips_files:

        with open(nips_file_path, "rb") as file:

            print(f"Opening file {nips_file_path}")

            for logical_record in nips.logical_record.read(file):

                record = fft.feed(logical_record)
                record_count = record_count + 1

                if isinstance(record, DataFileControlRecord):
                    print("Found DataFileControlRecord.")

    print(f"Total record count: {record_count}")


main(
    [
        "RG330.VNDB.NIPA68",
        "RG330.VNDB.NIPB68",
        "RG330.VNDB.NIPC68",
    ]
)

With this we can read a total of 129847 records.