Spectroscopy and Chemometrics/Machine Learning News Weekly #36, 2021

NIR Calibration-Model Services

With CM Service you can have customized optimized NIR calibrations developed without subscription. | NIRS NIR Spectroscopy ModelDevelopment MachineLearning Chemometrics LINK

Spectroscopy and Chemometrics News Weekly 35, 2021 | NIRS NIR Spectroscopy MachineLearning Spectrometer Spectrometric Analytical Chemistry Chemical Analysis Lab Labs Laboratories Laboratory Software IoT Sensors QA QC Testing Quality LINK

This week’s NIR news Weekly is sponsored by Your-Company-Name-Here – NIR-spectrometers. Check out their product page … link

Get the Spectroscopy and Chemometrics News Weekly in real time on Twitter @ CalibModel and follow us.




Near-Infrared Spectroscopy (NIRS)

“NIRS-ICA: A MATLAB Toolbox for Independent Component Analysis Applied in fNIRS Studies” | LINK

“NIRS and manure composition (NIMACO)” LINK

“Shining a light on cultural neuroscience: Recommendations on the use of fNIRS to study how sociocultural contexts shape the brain” LINK

“Evaluation of Grape Juice Quality by Near Infrared Spectroscopy and Chemometrics” “การ ประเมิน คุณภาพ น้ำ องุ่น ด้วย เนีย ร์ อินฟราเรด สเปก โทร ส โก ปี และ เค โมเม ท ริก ซ์” LINK

“Development of a soil fertility index using on-line Vis-NIR spectroscopy” LINK

“PDA@Ti3C2Tx as a novel carrier for pesticide delivery and its application in plant protection: NIRresponsive controlled release and sustained antipest activity” LINK

“NIR surface enhanced Raman spectra of biological hemes: Solvation and plasmonic metal effects” LINK




Infrared Spectroscopy (IR) and Near-Infrared Spectroscopy (NIR)

“Near Infrared Feature Waveband Selection for Fishmeal Quality Assessment by Frequency Adaptive Binary Differential Evolution” LINK

“Polymers : Prediction of Residual Curing Capacity of Melamine-Formaldehyde Resins at an Early Stage of Synthesis by In-Line FTIR Spectroscopy” LINK

“Infrared spectroscopy and forensic entomology: Can this union work? A literature review” | LINK

“Near-Infrared Spectroscopy for Ganoderma Boninense Detection: An Outlook” | LINK

“Investigation of domestic hen Egg quality in terms of Fertilization during storage using Near Infrared Spectroscopy” LINK

“Application of near-infrared spectroscopy in detecting residual crystallinity in carbamazepine-Soluplus® solid dispersions prepared with solvent casting and hot-melt …” LINK

“From at-line analysis to on-site control in the Iberian pig industry using Near Infrared Spectroscopy sensors” LINK

“Determination of α-guaiene and azulene chemical content in patchouli aromatic oil (Pogostemon cablin Benth.) from Indonesia by Near-infrared spectroscopy” LINK




Hyperspectral Imaging (HSI)

“Hyperspectral Image Stitching via Optimal Seamline Detection” LINK

“Optimization of a saccharin molecularly imprinted solid-phase extraction procedure and evaluation by MIR hyperspectral imaging for analysis of diet tea by HPLC” LINK

“Hyperspectral detection and monitoring of salt stress in pomegranate cultivars” LINK

“Supervoxel-Based Intrinsic Scene Properties From Hyperspectral Images and LiDAR” LINK

” A combination method of stacked autoencoder and 3D deep residual network for hyperspectral image classification” LINK




Chemometrics and Machine Learning

“Multi-product Calibration Model for Soluble Solids and Water Content Quantification in Cucurbitaceae family, using Visible/Near-Infrared Spectroscopy” LINK

“Deep chemometrics: Validation and transfer of a global deep nearinfrared fruit model to use it on a new portable instrument” LINK

“Healthcare : Multimodal Early Alzheimers Detection, a Genetic Algorithm Approach with Support Vector Machines” LINK

“Ability of near infrared spectroscopy and chemometrics to measure the phytic acid content in maize flour” LINK

“One-class classification of special agroforestry Brazilian coffee using NIR spectrometry and chemometric tools” LINK

“Remote Sensing : Comparison of Modelling Strategies to Estimate Phenotypic Values from an Unmanned Aerial Vehicle with Spectral and Temporal Vegetation Indexes” LINK




Facts

“Spectral Anomaly Detection Based on Dictionary Learning for Sea Surfaces” LINK




Research on Spectroscopy

“A Review on Meat Quality Evaluation Methods Based on Non-Destructive Computer Vision and Artificial Intelligence Technologies” | LINK

“Electrochemical and Spectroscopic Studies on Triarylamine‐Polychlorotriphenylmethyl Dyads with Particularly Strong Triarylamine Donors” LINK




Equipment for Spectroscopy

“Polymers : Polyethylene/Polyamide Blends Made of Waste with Compatibilizer: Processing, Morphology, Rheological and Thermo-Mechanical Behavior” LINK




Future topics in Spectroscopy

According to Precision Business Insights the global molecular spectroscopy market size valued USD 5.6 billion in 2020 and expected to reach USD 10.4 billion by 2027, at a CAGR of 7.23% during forecast period 2021 to 2027. LINK




Process Control and NIR Sensors

“Monitoring of high-load dose formulations based on co-processed and non co-processed excipients” LINK

“Sensors : Development of Gas Sensor Array for Methane Reforming Process Monitoring” LINK

“Process analytical technology as a tool to optimize and accelerate pharmaceutical process development” LINK

“Recent developments in vibrational spectral analyses for dynamically assessing and monitoring food dehydration processes” LINK




Environment NIR-Spectroscopy Application

“Effects of Biochar Addition Under Different Water Management Conditions on N 2 O Emission From Paddy Soils in Northern Hainan” LINK

“Remote Sensing : Using Synthetic Remote Sensing Indicators to Monitor the Land Degradation in a Salinized Area” LINK




Agriculture NIR-Spectroscopy Usage

“Development of Attenuated Total Reflectance Mid-Infrared (ATR-MIR) and Near-Infrared (NIR) Spectroscopy for the Determination of Resistant Starch Content in Wheat Grains” LINK

“Extraction of spatial-spectral homogeneous patches and fractional abundances for field-scale agriculture monitoring using airborne hyperspectral images” LINK

“Research advancements in optical imaging and spectroscopic techniques for nondestructive detection of mold infection and mycotoxins in cereal grains and nuts” LINK

“Agronomy : Characterization of Oilseed Crop Noug (Guizotia abyssinica) Using Agro-Morphological Traits” LINK

“Tradeoffs in the Spatial and Spectral Resolution of Airborne Hyperspectral Imaging Systems: A Crop Identification Case Study” LINK

“Agriculture : Quantitative Evaluation of Color, Firmness, and Soluble Solid Content of Korla Fragrant Pears via IRIV and LS-SVM” LINK

“Agronomy : Agro-Morphological and Molecular Variability among Algerian Faba Bean (Vicia faba L.) Accessions” LINK

“Effects of Dielectric Properties and Microstructures on Microwave-Vacuum Drying of Mushroom (Agaricus bisporus) Caps and Stipes Evaluated by Non-destructive …” LINK

“Liming improves the stability of soil microbial community structures against the application of digestate made from dairy wastes” LINK

“Animals : Identifying Health Status in Grazing Dairy Cows from Milk Mid-Infrared Spectroscopy by Using Machine Learning Methods” LINK

“Agronomy : Effect of Ascophyllum nodosum Alga Application on Microgreens, Yield, and Yield Components in Oats Avena sativa L” LINK




Horticulture NIR-Spectroscopy Applications

“… of the Soluble Solid Content and Acidity by Prediction Models for Different Colored Tomato Fruits using a Small Device for Visible and Near-infrared Spectroscopy …” LINK




Food & Feed Industry NIR Usage

“Foods : Study on the Lamb Meat Consumer Behavior in Brazil” LINK

“Foods : Analytical Rheology of Honey: A State-of-the-Art Review” LINK

“Nondestructive identification of barley seeds variety using near‐infrared hyperspectral imaging coupled with convolutional neural network” LINK

“Foods : Interaction of Bioactive Mono-Terpenes with Egg Yolk on Ice Cream Physicochemical Properties” LINK




Other

“Influence of modified biochar supported Fe-Cu/polyvinylpyrrolidone on nitrate removal and high selectivity towards nitrogen in constructed wetlands” LINK

“The Story of 5d Metallocorroles: From Metal-Ligand Misfits to New Building Blocks for Cancer Phototherapeutics” LINK

“Effect of Surface Chemical Modification on the Self Assembly of Metal Nanoparticles” LINK

“GEOGRAPHICAL IDENTIFICATION OF DURIAN (CV. MONTHONG) FROM PRACHUAP KIRI KHAN AND CHANTHABURI PROVINCE BY USING NEAR …” LINK

“On the signal contribution function with respect to different norms” LINK

“Forests : Bioclimatic Suitability of Actual and Potential Cultivation Areas for Jacaranda mimosifolia in Chinese Cities” LINK

“Effects of Vanadium on the Structural and Optical Properties of Borate Glasses Containing Er<sup>3+</sup> and Silver Nanoparticles” LINK





.

Spectroscopy and Chemometrics News Weekly #19, 2021

NIR Calibration-Model Services

Spectroscopy and Chemometrics News Weekly 18, 2021 | NIRS NIR Spectroscopy MachineLearning Spectrometer Spectrometric Analytical Chemistry Chemical Analysis Lab Labs Laboratories Laboratory Software IoT Sensors QA QC Testing Quality LINK

This week’s NIR news Weekly is sponsored by Your-Company-Name-Here – NIR-spectrometers. Check out their product page … link

Get the Spectroscopy and Chemometrics News Weekly in real time on Twitter @ CalibModel and follow us.





Near-Infrared Spectroscopy (NIRS)

“Integrated (1)H NMR fingerprint with NIR spectroscopy, sensory properties, and quality parameters in a multi-block data analysis using ComDim to evaluate coffee blends” LINK

“Efficient Nearinfrared Pyroxene Phosphor LiInGe2O6:Cr3+ for NIR Spectroscopy Application” LINK

“Transfer learning and wavelength selection method in NIR spectroscopy to predict glucose and lactate concentrations in culture media using VIPBoruta” LINK

“Theranostic Near-Infrared-Active Conjugated Polymer Nanoparticles” LINK

“Integrated NIRS and QTL assays reveal minor mannose and galactose as contrast lignocellulose factors for biomass enzymatic saccharification in rice” LINK

“Age estimation of barramundi (Lates calcarifer) over multiple seasons from the southern Gulf of Carpentaria using FT-NIR spectroscopy” | LINK

“Evaluation of Minimum Preparation Sampling Strategies for Sugarcane Quality Prediction by vis-NIR Spectroscopy” LINK

“Differentiation between Fresh and Thawed Cephalopods Using NIR Spectroscopy and Multivariate Data Analysis” LINK

“Foods, Vol. 10, Pages 885: Histamine Control in Raw and Processed Tuna: A Rapid Tool Based on NIR Spectroscopy” LINK




Infrared Spectroscopy (IR) and Near-Infrared Spectroscopy (NIR)

“RenalClearable NickelDoped Carbon Dots with Boosted Photothermal Conversion Efficiency for Multimodal ImagingGuided Cancer Therapy in the Second NearInfrared Biowindow” LINK

“Simultaneous Broadening and Enhancement of Cr3+ Photoluminescence in LiIn2SbO6 by Chemical Unit Cosubstitution: NightVision and NearInfrared Spectroscopy Detection Applications” LINK

“Applied Sciences, Vol. 11, Pages 3701: Measurement of Temperature and H2O Concentration in Premixed CH4/Air Flame Using Two Partially Overlapped H2O Absorption Signals in the Near Infrared Region” LINK

“Fourier-Transform Infrared Spectroscopy as a Discriminatory Tool for Myotonic Dystrophy Type 1 Metabolism: A Pilot Study” IJERPH LINK

“Application of machine learning to estimate fireball characteristics and their uncertainty from infrared spectral data” LINK

“Cross Target Attributes and Sample Types Quantitative Analysis Modeling of Near-infrared Spectroscopy Based on Instance Transfer Learning” LINK

“Fast at-line characterization of solid organic waste: Comparing analytical performance of different compact near infrared spectroscopic systems with different …” LINK

“On-line identification of silkworm pupae gender by short-wavelength near infrared spectroscopy and pattern recognition technology” LINK

“The Use of Multispectral Imaging and Single Seed and Bulk Near-Infrared Spectroscopy to Characterize Seed Covering Structures: Methods and Applications in Seed …” LINK

” The use of infrared reflectance spectroscopy to predict the dry matter intake of lactating grazing dairy cows” LINK

“Development of a Novel Green Tea Quality Roadmap and the Complex Sensory-associated Characteristics exploration using Rapid Near-Infrared Spectroscopy …” LINK

“… of the Neutral and Acid Detergent Fiber Fractions of Chickpea (Cicer arietinum L.) by Combining Modified PLS and Visible with Near-Infrared Spectroscopy” LINK

“Non-destructive estimation of fibre morphological parameters and chemical constituents of Tectona grandis Lf wood by near infrared spectroscopy” LINK

“NEAR INFRARED SPECTROSCOPY MEASUREMENT AND KINETIC MODELING FOR PHYSIOCHEMICAL PROPERTIES OF TABTIM FISH (HYBRID TILAPIA …” LINK

“A simple multiple linear regression model in near infrared spectroscopy for soluble solids content of pomegranate arils based on stability competitive adaptive re …” LINK

“Intelligent assessment of the histamine level in mackerel (Scomber australasicus) using near-infrared spectroscopy coupled with a hybrid variable selection strategy” LINK

“Portable Near Infrared Spectroscopy as a Tool for Fresh Tomato Quality Control Analysis in the Field” LINK

” Focused echocardiography, end-tidal carbon dioxide, arterial blood pressure or near-infrared spectroscopy monitoring during paediatric cardiopulmonary …” LINK

“EXPRESS: Fourier Transform Infrared (FT-IR) Imaging Analysis of Interactions Between Polypropylene Grafted with Maleic Anhydride (MAPP) and Silica Spheres (SS) …” LINK




Hyperspectral Imaging (HSI)

“Hazelnuts classification by hyperspectral imaging coupled with variable selection methods” LINK

“Towards the development of a sterile model cheese for assessing the potential of hyperspectral imaging as a non-destructive fungal detection method” LINK

“Geographical origin discriminant analysis of Chia seeds (Salvia hispanica L.) using hyperspectral imaging” LINK




Chemometrics and Machine Learning

“Bayesian subset selection and variable importance for interpretable prediction and classification. (arXiv:2104.10150v1 [stat.ML])” LINK

“MachineLearning and Feature Selection Methods for EGFR Mutation Status Prediction in Lung Cancer” LINK

“Remote Sensing, Vol. 13, Pages 1598: Development of Novel Classification Algorithms for Detection of Floating Plastic Debris in Coastal Waterbodies Using Multispectral Sentinel-2 Remote Sensing Imagery” LINK

“Sensors, Vol. 21, Pages 2871: A Novel Runtime Algorithm for the Real-Time Analysis and Detection of Unexpected Changes in a Real-Size SHM Network with a Quasi-Distributed FBG Sensors” LINK

“NIR spectroscopy coupled with chemometric algorithms for the prediction of cadmium content in rice samples” LINK

” Determination of petroleum hydrocarbon contamination in soil using VNIR DRS and PLSR modeling” LINK

” Establishment and applicant of near-infrared reflectance spectroscopy models for predicting protein, linolenic acid and lignan contents of flaxseed” LINK

“Detection of chlorpyrifos and carbendazim residues in the cabbage using visible/near-infrared spectroscopy combined with chemometrics” LINK

“A Model Based on Clusters of Similar Color and NIR to Estimate Oil Content of Single Olives” LINK

“The Organochlorine Pesticide Residues of Mesona Chinensis Benth by Near Infrared (NIR) Spectroscopy and Chemometrics” LINK

“Strategies for the Development of Spectral Models for Soil Organic Matter Estimation” Remote Sensing LINK

“THE QUANTIFICATION OF NEUROCOGNITIVE IMPAIRMENT ACROSS THE SPECTRUM OF KIDNEY DISEASE” LINK




Facts

“IJMS, Vol. 22, Pages 4347: Mitochondrial Bioenergetic, Photobiomodulation and Trigeminal Branches Nerve Damage, Whats the Connection? A Review” LINK




Research on Spectroscopy

“ndothelial and microvascular function in CKD: Evaluation methods and associations with outcomes” LINK




Equipment for Spectroscopy

“On-line monitoring of egg freshness using a portable NIR spectrometer in tandem with machine learning” LINK




Environment NIR-Spectroscopy Application

” Current sensor technologies for in situ and on-line measurement of soil nitrogen for variable rate fertilization-A review.” LINK

” Mid-Infrared Spectroscopy Supports Identification of the Origin of Organic Matter in Soils. Land 2021, 10, 215″ LINK




Agriculture NIR-Spectroscopy Usage

“Crystals, Vol. 11, Pages 458: Boron Influence on Defect Structure and Properties of Lithium Niobate Crystals” | LINK

“Remote Sensing, Vol. 13, Pages 1620: Estimating Plant Nitrogen Concentration of Rice through Fusing Vegetation Indices and Color Moments Derived from UAV-RGB Images” LINK

“Consensus rule for wheat cultivar classification on VL, VNIR and SWIR imaging” LINK

” Changes in the Milk Market in the United States on the Background of the European Union and the World” LINK

“High-Resolution Airborne Hyperspectral Imagery for Assessing Yield, Biomass, Grain N Concentration, and N Output in Spring Wheat” RemoteSensing LINK

” Near infrared hyperspectral imaging of the hemodynamic and metabolic states of the exposed cortex: in vivo investigation on small animal models” LINK

“Engineered Protein PhotoThermal Hydrogels for Outstanding In Situ Tongue Cancer Therapy” LINK




Food & Feed Industry NIR Usage

“The quality and shelf life of biscuits with cryoground proso millet and buckwheat byproducts” LINK

“Performance of different portable and hand-held near-infrared spectrometers for predicting beef composition and quality characteristics in the abattoir without meat sampling” LINK

“New Approaches to Detect Compositional Shifts in Fish Oils” LINK




Medicinal Spectroscopy

“Aplicação de espectroscopia no infravermelho próximo e análise multivariada para identificação e quantificação de hidrocarbonetos totais do petróleo em solo” | LINK




Other

“Vibrational Analysis of Benziodoxoles and Benziodazolotetrazoles” LINK

“Recent advances in Unmanned Aerial Vehicles forest remote sensing—A systematic review. Part II: Research applications” LINK

“Physical and thermal properties of gold nanoparticles embedded Nd3+-doped borophosphate glasses: Spectroscopic parameters” LINK

“Spectral Assessment of Organic Matter with Different Composition Using Reflectance Spectroscopy” Remote Sensing LINK




NIR-Predictor – Manual


NIR-Predictor – Manual

Predicting Spectra

It’s easy to use with NIR-Predictor,
just drag & drop your data for getting the prediction results.

It supports an automatic file format detection.
So you don’t need to specify the instrument type and settings! See the list of supported formats and NIR Vendors: NIR-Predictor supported Spectral Data File Formats

Use the included data to checkout how it feels:

  1. Open the demo Spectra folder by using the Menu > Open Demo Spectra or press F8.
    There are files with spectra from different Vendors.

  2. Drag & drop a spectra file onto the NIR-Predictor window (or press Ctrl+O as for ’Open some files).

  3. The spectra will be

    • loaded
    • pre-processed
    • predicted and
    • reported

Note:
All the steps are fully automatic.
All calibrations that are compatible with the spectra, will produce prediction results in one go.
To select specific calibrations choose the Application. Where the ” ” empty means use all the calibrations.
To define a Application read more in chapter “Applications”

Hint:
To get access to Statistics of Predictions and Reports use the Menu > Show more/less (Ctrl+M) or you can simply resize the window. Here you can also re-do the Analyze step manually with changed inputs (e.g. Result Ordering).


Creating your own Calibrations

How it works – step by step

  1. You have measured your samples with you NIR-Instrument Software.
    And got the Lab-values of these samples.

    samples
    -> measured NIR-spectra
    -> Lab-references analytics

  2. Now you need to combine these data.

    NIR-spectra + Lab-references
    -> PropertiesBySamples

    Note: If you combined these data already in your NIR software used,
    and you can export it as a JCAMP-DX file then use
    Menu > Create Request File .req ... (F2)
    and read the “Help.html” and NIR-Predictor JCAMP.
    Else proceed as below.

    The NIR-Predictor provides tooling for that:

    Menu > Create Properties File... (F6)

    Select the folder with your NIR spectra measured for an application.
    NIR-Predictor creates a customized Properties file template for that data to enter the Lab values.

    Note: You don’t need to specify your instrument or vendor or an application. It’s all done automatically. And also the sample spectra are detected and grouped automatically!

  3. Use your favorite editor or spreadsheet program to enter and copy&paste
    the Lab-references Values into the columns “Prop1”, “Prop2” etc. and save the file.

  4. A final check of your entered data is done by NIR-Predictor,
    to make sure your data ist complete and all is fine.

    Menu > Create Calibration Request... (F7)

    Select the folder with the filled file.
    A CalibrationRequest.zip is created with the necessary data
    if enougth diverse Lab values are entered.

  5. Email the CalibrationRequest.zip file
    to info@CalibrationModel.com to develop the calibrations.

  6. When your calibrations are ready, you will receive an email with a link
    to the CalibrationModel WebShop where
    you can purchase and download the calibration files,
    that work with our free NIR-Predictor software without internet access.

    Note: Your sent NIR data is deleted after processing.
    We do not collect your NIR data!

Note: Further details can be found under “Create Properties File” and “Create Calibration Request”.


Configure the Calibrations for prediction usage

Configuration:

  1. in NIR-Predictor : Menu > Open Calibrations (F9)

  2. an explorer window is opened where the calibrations are located

  3. create a folder for your application, choose a name

  4. copy the calibration file(s) (*.cm) into that folder

  5. in NIR-Predictor : Menu > Search and load Applications (F4)

Usage:

  1. in NIR-Predictor : open the Application drop down list, and select your application by name

  2. if all is fine, the calibration file is valid and not expired, it shows : Calibration “1 valid calibation”

  3. the NIR-Predictor is now ready to predict

  4. to switch the application, goto 6.


Applications

The Application concept allows to group multiple Calibrations together for an Application. By selecting an Application before prediction, only the Calibrations belonging to the Application will be used for Prediction. In the Demo Data this is used to have multiple spectrometer as Application. This can be used easily as e.g. as Application “Meat Products” containing Fat and Moisture Calibration.

To create an Application, create a folder with the Application’s name inside the Calibrations folder, and move/copy all the Calibrations files to this Application folder. To remove a Calibration from the Application, remove the Calibration file from the Application folder.

After creating an new Application folder, press menu Search and load Applications (F4) to update the NIR-Predictor dialog where the Application can be selected via the dropdown list. You don’t need to close the NIR-Predictor.

After moving Calibration files around, press menu Search and load Calibrations (F5) to update the NIR-Predictor dialog.

The use-all case

In the NIR-Predictor dialog where the Application can be selected via the dropdown list, the empty "" name means that all (yes all) valid Calibrations will be used for prediction.

Note: The Prediction Report will contain only results from spectral compatible Calibrations with the given spectra. That allows to automatically handle the multi vendor NIR instrument usage.


Prediction Result Report

Histograms of Prediction Values per Property

Shows the distribution of the predicted results per calibration. The histogram range contains the range of the calibrated property and includes the predicted results.

The histogram bar (bin) color is defined as follow:

  • blue : all predictions inside calibration range.
  • red : all predictions outside calibration range.
  • orange : some overlaps with calibration range.
    So not all spectra in a orange bin are outside calibration range.
Histograms

Note: Predicted values are always shown in Histogram table and Prediction Value List table, even if the spectrum does not fit into model (spectrum different to model, aka Residual Outlier) shown as Out = X.

Note: Old browsers like Microsoft Internet Explorer 11 don’t support the grafics for Histogram charts. Use an current browser like Firefox or Chrome or Edge.

Note: If your browser opens the report too slow, try to deactivate some browser plugins, because they can filter what you look at and some add-ons are really slow.

Spectra Plot Thumbnail on the Prediction Report

Visualizes the min,median,max spectrum of the spectra dropped as files on the NIR-Predictor. This gives a minimal and good spectral overview of the predicted property results.

  • Spectra Plot color legend: min,median,max spectrum by predicted property or if no calibration is available by spectral intensity.

  • The min,median,max is determined from the predicted properties or if not available from the intensity of the spectra.

  • Beside the histogram of the predicted properties, where the distribution can be seen, the spectra shown are the ones from min,median,max predicted property.

  • This gives a minimal and good spectral overview of the predicted property results.

  • The “Spectral Range” and number of datapoints is shown in the Prediction Report Header below the listed spectra files.

  • To zoom the spectra plot a little, zoom the report in the browser (hold ctrl + mouse wheel, or pinch on touch screen).

  • The spectra plots and histograms are stored with the report and can be archived.

Note

  • Note that the spectra are shown in the raw values that are loaded, they are not shown pre-processed as the calibration model uses them to make the prediction.

  • Note that the median property spectrum is the median from the predicted property pobulation and not the “median” of the calibration property range.

  • Note that in the multi calibration prediction case, the spectra are selected for each property based on the related predicted property values and so the spectra plots shows typical different spectra.

Spectra Plot

Outlier Detection

To safeguard the prediction results, outliers are automatically checked for each individual prediction. This is based on limits that are determined when creating the calibration with the base data. Thus, a strange spectral measurement can be detected and signaled as an outlier even without base data only by means of the calibration and the NIR predictor. A prediction result with outlier warning is to be distrusted. How the various outlier tests are interpreted and how to avoid them in practice is described here.

The spectrum is an outlier to the model, if the spectrum is not similar with the spectra and lab-values the model is built with.

This legend is shown on each NIR-Predictor prediction report below the results:

Outlier (Out) Symbol Description

  • “X” : spectrum does not fit into model (spectrum different to model)
  • “O” : spectrum is wide outside model center (spectrum similar to model but far away)
  • “=” : prediction is outside upper or lower range of model (property outside model range)
  • “-” : spectrum is incompatible to calibration

Note: A prediction result with outlier warning is to be distrusted.

There are 3 outlier cases (X, O, =) and the incompatible data case “-”.

  • The bad case is “X”
  • the medium case is “O”
  • and the soft case is “=”.

The technical names in literature correspond to:

  • “X” : Spectral Residual Outlier
  • “O” : Leverage Outlier
  • “=” : Property Range Outlier

These 3 outlier cases can appear in combinations, like “XO=” or “XO” or “O=” or “X=”. The more outlier marker are shown the more likely the spectrum is an Outlier.

The default setting in NIR-Predictor Menu > “Report with Simplified Outlier Symbols”

  • is ON, that will show only the worst case instead of all combinations to have a simplified minimal information.
  • if OFF, that will show the combinations (e.g. “XO=” or “XO” or “O=” or “X=”), which is more informative for analyzing problem cases.

Some hints to avoid these Outliers:

  • “X” : spectrum does not fit into model (spectrum different to model)
    Check if the spectrum is noise only, or has no proper signal. That can happen when measured past the sample or measured into the air or at a different substance. If you have multiple NIR instruments of the same type, use spectra measured with different instruments for the calibration.

  • “O” : spectrum is wide outside model center (spectrum similar to model but far away) Sample temperature has an effect on NIR spectra shape, use spectra measured at different (typical use) temperatures (sample temperature, instrument temperature).

  • “=” : prediction is outside upper or lower range of model (property outside model range)
    Use more spectra for the calibration in the Lab value region where your special interest is. If the predicted value is only a little bit out of the calibration range, it can be Ok. Add these spectra to the calibration spectra (with the Lab values), to extend the prediction range of the calibration.

  • “-” : spectrum is incompatible to calibration
    The spectra (from the NIR instrument) has a different wavelength range or a different resolution than the spectra used for calibration. Check Instrument settings (wavelength range, resolution)

Result Ordering

To change the ordering, a drop-down-box is located below the Analyze button. If there is an analysis from the current session, and the Result Ordering is changed, the data is re-Analyzed and reported with the new Result Ordering setting. That allows to compare the different orderings. The Result Ordering is listed in the Prediction Report above the Prediction Value List and stored in the settings.

The order/sorting of the prediction results of the spectra can be defined:

  • GivenOrder (default) the given order of the spectra from file select dialog or drag&drop

*) sorted : ascending sort

  • Date_Name sorted by Date (if any) and then by Name
  • Name_Date sorted by Name and then by Date
  • Date_NamesWithNumbers sorted by Date (if any) and then by Name with number logic
  • NamesWithNumbers_Date sorted by Name with number logic (e.g. “ABC1” is before “ABC002” ) and then by Date

*) as above but sorted Rev : reverse sort = descending sort

  • Rev_Date_Name
  • Rev_Name_Date
  • Rev_Date_NamesWithNumbers
  • Rev_NamesWithNumbers_Date

E.g. with reverse sort by Rev_Date_Name, the newest spectra appear on top.

Depending on how many calibrations are used the result table is getting broader. To print the report (e.g. to Adobe PDF, FreePDF or Microsoft XPS), sometimes the landscape format is shorter in number of pages or in portrait a scale of 80% fits nicely. Or try another internet browser (Mozilla Firefox, Google Chrome, Microsoft Edge, …) to print the report and set the browser as your default browser so it will be opened by default.

Archiving Reports

Each report is contained in one file only, including the grafics. To save storage space the report file folder can be compressed to a zip file (.zip, .7z).


Enter lab values to NIR spectra

Entering the laboratory reference values for NIR calibrations

We have developed specialized tools into NIR-Predictor to combine the NIR and Lab data is a sample-based safe manner.

The main target is to improve Data Quality during the step of combining of the Lab data and the NIR data, because to model a good reliable calibration the data that build the base needs to be of high quality.

It also simplifies to enter the lab values manually to the corresponding NIR data, because of automatically grouping repeated NIR measurements of the same sample, so the lab values can be entered sample based and not by spectrum.

It helps to avoid false reference data, because of the broken relation of NIR spectra and reference values, data entry on the wrong position in the table.

And Helps to detect errors of duplicated or multiple copies of spectra files, and checks for inconsistencies in Date-Time and Sample-Naming. It also checks for missing values.

That all increases the Data Quality for the next step of Calibration Development, and makes data entry a less time consuming and less risky work.

How it works

  1. Menu > Create Properties File... (F6) select the folder with your NIR spectra measured for an application. NIR-Predictor creates a Properties file template for that data : PropertiesBySamples.csv.txt

  2. Use your favorite editor or spreadsheet program to enter and copy&paste the Lab Values into the columns and save the file.

  3. Menu > Create Calibration Request... (F7) select the folder with the filled file for a last check and a Calibration Request file is created with the needed data as a single zip file.

  4. Email the Calibration Request file to info@CalibrationModel.com to develop the calibrations.

Ok that is it, the NIR-Predictor guides you through the steps needed. And if you need to know more details, the Chapter “Create Properties File” is for you.

Create Properties File

Note:

  • If you have (exported) JCAMP-DX files containing the Lab-Values, you don’t need to do this step.
    You can send the JCAMP file with your Request (.req) file directly to the calibration service at info@CalibrationModel.com.
  • If your JCAMP-DX files does NOT contain Lab-Values, this is a way to go.

For calibrating the spectra to the lab-values you need to assign the lab-values to the spectra. The easiest way is to have a table where each spectrum (row) is linked to multiple lab-values (columns). This function Create Properties File build such a table for the selected spectra folder automatically!

This table is stored in the file PropertiesBySamples.csv.txt. This can be created for any spectra folder you like. The file extension is .csv.txt to make it easy to edit in a text editor and also in a spreadsheet (excel). The columns are standard TAB separated.

The file header line contains :

Sample Replicates Names Prop1 Prop2 Prop3 DateFirst DateLast Hashes

Where Name and Date describes the spectrum.

Prop1, Prop2, Prop3 are the place to enter the Lab Reference Concentrations properties corresponding to each spectrum. It can be extended to Prop4, Prop5, … etc. Of course you can enter real word names like “Fat (%)” instead of “Prop1”. It’s recommended to put the measurement unit beside the name.

Replicates is the number of replicated or repeated spectra of a sample that is grouped together in the Sample based property file. Sample name and the DateFirst / DateLast between the sample spectra are measured.

Date format is ISO-8601. Missing Dates are 0002-02-02T00:00:00.0000000.

If the file PropertiesBySamples.csv.txt already exist in the selected folder, the user will be notified (it will not be overwritten, because the file may contain user entered Lab-values). The Lab Reference Concentrations values are initialized to 0 (zero) and needed to be changed.

Note: 0 is not interpreted as missing value! If you have a 0 concentration value, put in 0 or 0.0 .

The entry of properties is as easy as possible, because it’s organized by Sample (and not by Spectra), so it’s like your Lab-Value Table that is sample based. The sample rows are sorted in a special way by Sample name. Sorting by Date or alphabetically by Sample can done easily in a spreadsheet program.

Note: when coping lab values to the samples make sure they correspond, so that there are no gaps and the sorting is the same.

The Spectra (rows) are initially sorted by name (and date) to have the replicates/repeats together. You can sort for your convenience in a spreadsheet program.

Enter the Lab Reference Concentrations to the spectra/sample.

Enter the Lab-Values in spreadsheet (e.g. Excel) or a text editor (e.g. Notepad++). If done, use the next menu Create Calibration Request.

Hints: Data handling:

  • The NIR-Predictor creates the PropertiesBySamples.csv.txt once, after that the user is responsible for its content. That means NIR-Predictor does not change this file anymore.

  • You can remove entire rows (spectra) in the property file. You don’t need to remove the spectra files. The NIR-Predictor is aware of the relation, the PropertiesBySamples.csv.txt defines what will be calibrated.

  • How to add more spectra files?

    The additional spectra can be handled in a separate folder, create the property file and copy the spectra to the other folder and copy/merge the property files together in your editor or spreadsheet.

    Or

    Copy the spectra into the folder, rename the PropertiesBySamples.csv.txt to e.g. “PropertiesBySamples-Part1.csv.txt” and use Create Properties File to create a new PropertiesBySamples.csv.txt with all the spectra. You can copy/merge the content of the Properties files together in your editor or spreadsheet.

  • What happens with possible duplicate rows? It does no harm to the Calibration because we do an exact checking and data cleaning in the calibration process.

  • What happens to duplicate spectra names? The spectra names are not relevant for the calibration process. The spectra names are helpful to assign the lab-values to the corresponding spectrum entry. That’s why the table is initially sorted by name. The spectra names can be edited by the user.


Create Calibration Request

The menu function Create Calibration Request packs a created Properties file (see “Create Properties File”) and it’s linked spectra files in a compressed ZIP file for sending to the CalibrationModel.com Service.

Please note that the number of measured quantitative samples need to be at least 60 . That means you need at least 60 different spectra (not counting the replicate/repeated measurements).

It shows additional property information about the data you have entered, like – the property type (Quantitative) – it’s range (min – max) and – the number of unique values and – if the Lab-values are enough diverse to get calibrated.

First select the folder with the PropertiesBySamples.csv.txt and measured spectra files of samples you have Lab-values. The data is checked and you get notified what is missing or might be wrong. If something needs to be changed, edit the PropertiesBySamples.csv.txt and do Create Calibration Request again. Your last selected folder is remembered, so you can press return in the folder selection dialog.

Hint: The keyboard shortcuts for redoing it after you edited some entries is : F7 Return – that allows you to get the property information quickly.

Hint: If you open the PropertiesBySamples.csv.txt in a spreadsheet program, you can create Histogram plots of the entered Lab-values, to see in which range are to less samples measurements.

When all is fine

When all is fine the “CalibrationRequest.zip” file is created for that data.

The ZIP file contains:

  • your PropertiesBySamples.csv.txt
  • your personal REQuest file for your computer system, that looks like
    e.g. “337dcdc06b2d6dfb0b5c4bba578642312edf2ae84d909281624d7e26283e8b07 WIN-GB0PB48GSK4.req”
  • the spectra data files

Note: If the CalibrationRequest.zip file is already created and you change the PropertiesBySamples.csv.txt make sure to delete the old CalibrationRequest.zip file first! In the dialog it states if it was successfully created or NOT because it already exist. So you are always on the safe side.

Note: CalibrationRequest.zip file name contains the property names to know what would be calibrated and at the end an identification number for referencing the file. E.g. “CalibrationRequest ‘Prop1’ – ‘Prop2’ h31T3wOH.zip”


Program Settings

  • The users program settings are stored in UserSettings.json
  • The program counters are stored in GlobalCounters.json

Further References

NIR-Predictor Download

The free NIR-Predictor software
  • comes with demo data, so you can predict sample spectra with demo calibrations.
  • has no functional limitations, no nagging, no ads and needs no license-key.
  • you need no account and no registration to download and use.
  • runs on Microsoft Windows 10/8/7 (Starter, Basic, Professional) (32 bit / 64 bit).
  • no data is ever transmitted from your local machine. We don’t even collect usage data.
See more Videos



Beside the free NIR-Predictor software with Windows user interface,
the real-time Predictor Engine is also available
  • for embedded integration in application, cloud and instrument-software (ICT).
  • As a light-weigt single library file (DLL)
    with application programming interface (API),
    documentation and software development kit (SDK)
    including sample source code (C#).
  • Easy integration and deployment,
    no software license protection (no serial key, no dongle).
  • Put your spectrum as an array into the multivariate predictor,
    no specific file format needed.
  • Fast prediction speed and low latency
    because of compiled code library (direct call, no cloud API).
  • Protected prediction results with outlier detection information.
See NIR Method Development Service for Labs and NIR-Vendors (OEM, White-Label)



Software Size Date Comment
NIR-Predictor V2.6.0.2 (download)

What’s new, see Release Notes

By downloading and/or using the software
you accept the Software License Agreement (EULA)
3.7 MB 18.08.2021 public release

Minimal System Requirements
Windows 7 Starter 32Bit, 1.6 GHz, 2 GB RAM, non-Administrator account

Installation
There are no administrator rights required, unpack the zip file to a folder “NIR-Predictor” in your documents or on your desktop.
Read the ReadMe.txt and double click the NIR-Predictor.exe file.

Upgrade
If you have installed an older version of NIR-Predictor then unpack into a different folder named e.g. “NIR-PredictorVx.y”. All versions can run side-by-side. Copy the Calibrations in use to the new version into the “Calibration” folder. That’s all.

Uninstall
Make sure to backup your reports and calibrations inside your “NIR-Predictor” folder. Delete the “NIR-Predictor” folder.


Start Calibrate

See also: