For working with ENVI files I normally use GDAL as code can then be applied to different formats. However, there are a couple of limitations with GDAL when working with hyperspectral data in ENVI format:
- GDAL doesn’t copy every item from the header file to a new header file if they don’t fit in with the GDAL data model. Examples are FWHM and comments. Sometimes extra attributes are copied to the aux.xml file GDAL creates, these files aren’t read by ENVI or other programs based on IDL (e.g., ATCOR).
- For data stored Band Interleaved by Line (BIL) rather than Band Sequential (BSQ) reading and writing a band at a time is inefficient as it is necessary to keep jumping around the file
To overcome these issues NERC-ARF-DAN use their own Python functions for reading / writing header files and loading BIL files a line at a time. These functions have been tidied up and released through the NERC-ARF Tools repository on GitHub (https://github.com/pmlrsg/arsf_tools). The functions depend on NumPy.
To install them it is recommended to use the following steps:
- Download miniconda from http://conda.pydata.org/miniconda.html#miniconda and follow the instructions to install.
- Open a ‘Command Prompt’ / ‘Terminal’ window and install numpy by typing:
conda install numpy
- Download ‘arsf_tools’ from GitHub (https://github.com/pmlrsg/arsf_tools/archive/master.zip)
- Unzip and within a ‘Command Prompt’ or ‘Terminal’ window navigate to the the folder using (for example):
cd Downloads\arsf_tools-master
- Install the tools and library by typing:
python setup.py install
Note, if you are using Linux you can install the arsf_binary reader from https://github.com/arsf/arsf_binaryreader which is written in C++. The ‘arsf_envi_reader’ module will import this if available as it is faster than the standard NumPy BIL reader.
If you are a UK based researcher with access to the JASMIN system the library is already installed and can be loaded using:
module load contrib/arsf/arsf_tools
An simple example of reading each line of a file, adding 1 to every band and writing back out again is:
from arsf_envi_reader import numpy_bin_reader
from arsf_envi_reader import envi_header
# Open file for output
out_file = open("out_file.bil", "w")
# Open input file
in_data = numpy_bin_reader.BilReader("in_file.bil")
for line in in_data:
out_line = line + 1
out_line.tofile(out_file)
# Copy header
envi_header.write_envi_header("out_file.bil.hdr",
in_data.get_hdr_dict())
# Close files
out_file.close()
in_data = None
A more advanced example is applying a Savitzky-Golay filter to each pixel. As the filter requires every band for each pixel it is efficient to work with BIL files.
For developing algorithms using spatial data, in particular multiple files it is recommended to convert the files to another band-sequential format using the ‘gdal_translate’ so they can be read using programs which use GDAL, for example RIOS or RSGISLib.
To convert files from ENVI BIL to ENVI BSQ and copy all header attributes the ‘envi_header’ module can be used after converting the interleave with GDAL. For example:
import os
import subprocess
from arsf_envi_reader import envi_header
# Set input and output image (could get with argparse)
input_image = "input.bil"
output_image = "output_bsq.bsq"
# Get interleave from file extension
output_interleave = os.path.splitext(output_image)[-1].lstrip(".")
# 1. Convert interleave
print("Converting interleave to {}".format(output_interleave.upper()))
gdal_translate_cmd = ["gdal_translate",
"-of", "ENVI",
"-co", "INTERLEAVE={}".format(output_interleave.upper())]
gdal_translate_cmd.extend([input_image, output_image])
subprocess.call(gdal_translate_cmd)
# 2. Copy header (GDAL doesn't copy all items)
# Find header files
input_header = envi_header.find_hdr_file(input_image)
output_header = envi_header.find_hdr_file(output_image)
# Read input header to dictionary
input_header_dict = envi_header.read_hdr_file(input_header)
# Change interleave
input_header_dict["interleave"] = output_interleave.upper()
# Write header (replace one generated by GDAL)
envi_header.write_envi_header(output_header, input_header_dict)
spectraldifferences 