NIR Analysis in Laboratory and Laboratories – aka NIR Labs and NIR testing


Do you have a NIR spectrometer in your Lab?

How many other analytics you do in the Lab could be done faster and cheaper with NIR?

Is this possible and precise enough?

Try, we have the solution for you!
You have the NIR, scan the samples, you have the lab values and the spectra, we calibrate for you!

To see if the application is possible and how precise it can be due to knowledge based intensive model optimizations.

We do the NIR feasibility study with data for you. Fixed prices

NIR has huge application potentials and it’s a Green analytical method, that’s fast and easy to use. And has today the possibility to scale out with inexpensive mobile NIR spectrometers.

Bring the Lab to the sample. To avoid sample transport and get immediate results for decision at the place or in the process.

Just try the NIR application, use the NIR daily, collect data in parallel, we develop, optimize and maintain the calibration models for you.

How do you think?

Start Calibrate


What is possible today with NIR?
The number of different Applications exploded in the last 2-3 years!
See NIR research papers news daily on @CalibModel or the 7-day summariesNIR News Weekly” here.

Spectroscopy and Chemometrics News Weekly #24, 2020

NIR Calibration-Model Services

Machine Learning for NIR Spectroscopy as a Service, a Game Changer for Productivity and Accuracy/Precision! Use the free NIR-Predcitor software to combine NIRS + Lab data and send your Calibration Request. LabManager Analysis MachineLearning LINK

“Food quality digitized at the “speed of light” ” : Food Sample -> measured with a NIRS spectrometer -> spectral data -> ⚖️ predicted with a NIRPredictor & CalibrationModel -> % quantitative results -> quality decision -> LINK

Spectroscopy and Chemometrics News Weekly 23, 2020 | NIRS NIR Spectroscopy MachineLearning Spectrometer Spectrometric Analytical Chemistry Chemical Analysis Lab Labs Laboratories Laboratory Software IoT Sensors QA QC Testing Quality LINK

Spektroskopie und Chemometrie Neuigkeiten Wöchentlich 23, 2020 | NIRS NIR Spektroskopie MachineLearning Spektrometer IoT Sensor Nahinfrarot Chemie Analytik Analysengeräte Analysentechnik Analysemethode Nahinfrarotspektroskopie Laboranalyse LINK

Spettroscopia e Chemiometria Weekly News 23, 2020 | NIRS NIR Spettroscopia MachineLearning analisi chimica Spettrale Spettrometro Chem IoT Sensore Attrezzatura analitica Laboratorio analisi prova qualità Analysesystem QualityControl LINK




Near-Infrared Spectroscopy (NIRS)

“Fiber Content Determination of Linen/Viscose Blends Using NIR Spectroscopy” LINK

“Characterization of a high power time-domain NIRS device: towards faster and deeper investigation of biological tissues” LINK

“… chamosite from an hydrothermalized oolitic ironstone (Saint-Aubin-des-Châteaux, Armorican Massif, France): crystal chemistry, Vis-NIR spectroscopy (red variety) and …” LINK

“Study on evolution methods for the optimization of machine learning models based on FT-NIR spectroscopy” LINK

“Vibrational coupling to hydration shell – Mechanism to performance enhancement of qualitative analysis in NIR spectroscopy of carbohydrates in aqueous environment.” LINK

” RAPID EVALUATION OF DRY WHITE KIDNEY BEANS COOKING CHARACTERISTICS BY NEAR-INFRARED (NIR) SPECTROSCOPY” LINK

For food analysts, how to choose between a ‘classic’ method and a ‘modern’ technique such as FT-NIR or RMN? Our recently available paper tries to answer that question based on error evaluation: LINK

“FT-NIR combined with chemometrics versus classic chemical methods as accredited analytical support for decision-making: application to chemical compositional compliance of feedingstuffs” LINK




Infrared Spectroscopy (IR) and Near-Infrared Spectroscopy (NIR)

“Functional Classification of Feed Items in Pampa Grassland, Based on Their Near-Infrared Spectrum” LINK

“A Rapid and Nondestructive Approach for the Classification of Different-Age Citri Reticulatae Pericarpium Using Portable Near Infrared Spectroscopy” LINK

“Near-infrared spectroscopy as a new method for post-harvest monitoring of white truffles” LINK

“Rapid Prediction of Apparent Amylose, Total Starch, and Crude Protein by Near‐Infrared Reflectance Spectroscopy for Foxtail Millet (Setaria italica)” LINK

“New Induced Mutation Genetic Algorithm for Spectral Variables Selection in Near Infrared Spectroscopy” LINK

“Quantification of Plant Root Species Composition in Peatlands Using FTIR Spectroscopy” LINK

“Functional classification of feed items in pampa grassland, based on their near-infrared spectrum” LINK

“A feasibility of nondestructive rapid detection of total volatile basic nitrogen content in frozen pork based on portable near-infrared spectroscopy” LINK

” Machine Learning Classification of Articular Cartilage Integrity Using Near Infrared Spectroscopy” LINK

“Has the time come to use near-infrared spectroscopy in your science classroom?” LINK

“Feasibility of using near-infrared measurements to detect changes in water quality” LINK

“A novel CC-tSNE-SVR model for rapid determination of diesel fuel quality by near infrared spectroscopy” LINK

“Optimizing analysis of coal property using laser-induced breakdown and near-infrared reflectance spectroscopies” LINK

“Probing Active Sites and Reaction Intermediates of Electrocatalysis Through Confocal Near-Infrared Photoluminescence Spectroscopy: A Perspective.” LINK

“Determination of in situ ruminal degradation of phytate phosphorus from single and compound feeds in dairy cows using chemical analysis and near-infrared spectroscopy” LINK

“Non-destructive assessment of moisture content and modulus of rupture of sawn timber Hevea wood using near infrared spectroscopy technique” LINK

“Accurate prediction of glucose concentration and identification of major contributing features from hardly distinguishable near-infrared spectroscopy” LINK

” Multiblock PLS-DA on fecal and plasma visible-near-infrared spectra for discriminating young bulls according to their efficiency. Preliminary results” LINK

“Examining the Utility of Visible Near-Infrared and Optical Remote Sensing for the Early Detection of Rapid ‘Ōhi‘a Death” LINK

“Developing deep learning based regression approaches for determination of chemical compositions in dry black goji berries (Lycium ruthenicum Murr.) using near-infrared hyperspectral …” LINK

” RAPID, NONDESTRUCTIVE AND SIMULTANEOUS PREDICTIONS OF SOIL CONTENT IN WULING MOUNTAIN AREA USING NEAR INFRARED …” LINK




Raman Spectroscopy

“Differentiating cancer cells using Raman spectroscopy (Conference Presentation)” LINK

“Applied Sciences, Vol. 10, Pages 3545: Raman Spectral Analysis for Quality Determination of Grignard Reagent” LINK

“Surfaceenhanced Raman spectroscopy for onsite analysis: A review of recent developments” LINK




Hyperspectral Imaging (HSI)

“Estimating leaf mercury content in Phragmites australis based on leaf hyperspectral reflectance” LINK

“A hyperspectral microscope based on a birefringent ultrastable common-path interferometer (Conference Presentation)” LINK

“Hyperspectral imaging of beet seed germination prediction” LINK

“Hyperspectral imaging for discrimination of Keemun black tea quality categories: Multivariate calibration analysis and data fusion” LINK

“Potential of deep learning and snapshot hyperspectral imaging for classification of species in meat” LINK

“Performance of Fluorescence and Diffuse Reflectance Hyperspectral Imaging for Characterization of Lutefisk: A Traditional Norwegian Fish Dish” LINK




Spectral Imaging

“Identify the ripening stage of avocado by multispectral camera using semi-supervised learning on small dataset” LINK

“Multispectral imaging for predicting the water status in mushroom during hotair dehydration” LINK




Chemometrics and Machine Learning

“Sample selection, calibration and validation of models developed from a large dataset of near infrared spectra of tree leaves” Eucalyptus forage quality LINK

“Determination of Loline Alkaloids and Mycelial Biomass in Endophyte-Infected Schedonorus Pratensis by Near-Infrared Spectroscopy and Chemometrics” LINK

“Detection and Assessment of Nitrogen Effect on Cold Tolerance for Tea by Hyperspectral Reflectance with PLSR, PCR, and LM Models” LINK

“Application of vibrationnal spectroscopy and chemometrics to access the quality of Locally produced antimalarial medicines in the Democratic Republic of Congo.” LINK

“Predicting total petroleum hydrocarbons in field soils with VisNIR models developed on laboratoryconstructed samples” LINK

“National spectral data and learning algorithms for potentially toxic elements modelling in forest soil horizons” LINK

“Rapid determination of the textural properties of silver carp (Hypophthalmichthys molitrix) using near-infrared reflectance spectroscopy and chemometrics” LINK

“Vibrational spectroscopy and chemometrics for quantifying key bioactive components of various plum cultivars grown in New Zealand” LINK




Equipment for Spectroscopy

“NearInfrared Multipurpose LanthanideImaging Nanoprobes” LINK




Process Control and NIR Sensors

“Non-invasive measurement of quality attributes of processed pomegranate products” LINK




Environment NIR-Spectroscopy Application

“Spectral Feature Selection Optimization for Water Quality Estimation.” LINK

“Remote Sensing, Vol. 12, Pages 931: Optical Water Type Guided Approach to Estimate Optical Water Quality Parameters” LINK

“Estimation of total nitrogen and organic carbon contents in mine soils with NIR reflectance spectroscopy and various chemometric methods” LINK




Agriculture NIR-Spectroscopy Usage

“Development of a compact multimodal imaging system for rapid characterisation of intrinsic optical properties of freshly excised tissue (Conference Presentation)” LINK

“Agriculture, Vol. 10, Pages 181: Grafting and ShadingThe Influence on Postharvest Tomato Quality” LINK

“Remote Sensing, Vol. 12, Pages 940: Editorial for the Special Issue Estimation of Crop Phenotyping Traits using Unmanned Ground Vehicle and Unmanned Aerial Vehicle Imagery”” LINK

“Predicting grain protein content of field-grown winter wheat with satellite images and partial least square algorithm.” LINK

“The development of models to predict the nutritional value of feedstuffs and feed mixture using NIRS” LINK

“Permafrost soil complexity evaluated by laboratory imaging Vis‐NIR spectroscopy” LINK




Horticulture NIR-Spectroscopy Applications

“Recent advances in imaging techniques for bruise detection in fruits and vegetables” LINK




Forestry and Wood Industry NIR Usage

“Nutritional characterization of trees from the Amazonian piedmont, Putumayo department, Colombia” LINK




Food & Feed Industry NIR Usage

“Approaches, applications, and future directions for hyperspectral vegetation studies: An emphasis on yieldlimiting factors in wheat” LINK

“Beef Nutritional Quality Testing and Food Packaging” LINK




Laboratory and NIR-Spectroscopy

“UV Irradiation and Near Infrared Characterization of Laboratory Mars Soil Analog Samples: the case of Phthalic Acid, Adenosine 5′-Monophosphate, L-Glutamic Acid …” molecular biosignatures; spectroscopy; lifedetection LINK




Other

LINK

“Effect of substrate temperature on the microstructural and optical properties of RF sputtered grown ZnO thin films” LINK

Using near-infrared light to 3-D print an ear inside the body LINK

“Eco-friendly dye sensitized solar cell using natural dye with solid polymer electrolyte as hole transport material” solarcell LINK





Spectroscopy and Chemometrics News Weekly #23, 2020

NIR Calibration-Model Services

New Free NIR-Predictor V2.6 software is released – New : reads and predicts also *.spc spectra file format (Thermo-Scientific / Galactic GRAMS) – Spectra Plots on the Prediction Reports NIRS NIR Spectroscopy Spectrometer QualityControl Lab Laboratory Analysis LINK
Spectra Plot


Spectroscopy and Chemometrics News Weekly 22, 2020 | NIRS NIR Spectroscopy MachineLearning Spectrometer Spectrometric Analytical Chemistry Chemical Analysis Lab Labs Laboratories Laboratory Software IoT Sensors QA QC Testing Quality LINK

Spektroskopie und Chemometrie Neuigkeiten Wöchentlich 22, 2020 | NIRS NIR Spektroskopie MachineLearning Spektrometer IoT Sensor Nahinfrarot Chemie Analytik Analysengeräte Analysentechnik Analysemethode Nahinfrarotspektroskopie Laboranalyse LINK

Spettroscopia e Chemiometria Weekly News 22, 2020 | NIRS NIR Spettroscopia MachineLearning analisi chimica Spettrale Spettrometro Chem IoT Sensore Attrezzatura analitica Laboratorio analisi prova qualità Analysesystem QualityControl LINK

This week’s NIR news Weekly is sponsored by Your-Company-Name-Here – NIR-spectrometers. Check out their product page … link

Get the Chemometrics and Spectroscopy News in real time on Twitter @ CalibModel and follow us.




Near-Infrared Spectroscopy (NIRS)

“Potential of Vis-NIR spectroscopy for detection of chilling injury in kiwifruit” LINK

“The application of NIR spectroscopy in moisture determining of vegetable seeds” LINK

“Detection and quantification of active pharmaceutical ingredients as adulterants in Garcinia cambogia slimming preparations using NIR spectroscopy combined with …” LINK

“Vibrational coupling to hydration shell–Mechanism to performance enhancement of qualitative analysis in NIR spectroscopy of carbohydrates in aqueous environment” LINK

“Determination of metmyoglobin in cooked tan mutton using Vis/NIR hyperspectral imaging system” LINK

“The past, present, and prospective on UV-VIS-NIR skin photonics and spectroscopy-a wavelength guide.” LINK

“Differentiation of South African Game Meat Using Near-Infrared (NIR) Spectroscopy and Hierarchical Modelling” LINK

“Study on Detection Methods for Frying Times of Soybean Oil Based on NIRS” LINK

“Non-destructive Detection the Content of Acid Detergent Fiber in Corn Stalk Using NIRS” LINK

“Changes in chemical components with NIR spectroscopy and durability of samama wood treated with boron, methyl methacrylate and heat treatment” LINK

“Principle Component Analysis (PCA)-Classification of Arabica green bean coffee of North Sumatera Using FT–NIRS” LINK




Infrared Spectroscopy (IR) and Near-Infrared Spectroscopy (NIR)

“Near-infrared wavelength-selection method based on joint mutual information and weighted bootstrap sampling” LINK

“Rapid and simultaneous analysis of direct and indirect bilirubin indicators in serum through reagent-free visible-near-infrared spectroscopy combined with …” LINK

“Fast Detection Method of Antarctic Krill Meat Quality Based on Near Infrared Spectroscopy” LINK

“Sensors, Vol. 20, Pages 1472: Fusion of Mid-Wave Infrared and Long-Wave Infrared Reflectance Spectra for Quantitative Analysis of Minerals” LINK

“Determination of pectin content in orange peels by Near Infrared Hyperspectral Imaging” LINK

“Near-infrared spectroscopy as a quantitative spasticity assessment tool: A systematic review.” LINK

“Determination of nutritional parameters of bee pollen by Raman and infrared spectroscopy.” LINK

“Detection of aflatoxin B1 on corn kernel surfaces using visible-near infrared spectroscopy” LINK

“Soil NPK Levels Characterization Using Near Infrared and Artificial Neural Network” LINK

” Estimation of moisture in wood chips by Near Infrared Spectroscopy” LINK

“Biosensors, Vol. 10, Pages 41: Rapid Nondestructive Detection of Water Content and Granulation in Postharvest Shatian Pomelo Using Visible/Near-Infrared Spectroscopy” LINK

“Prognostic value of near-infrared spectroscopy in hypoxic-ischaemic encephalopathy” LINK




Raman Spectroscopy

“Quantitative models for detecting the presence of lead in turmeric using Raman spectroscopy” LINK




Hyperspectral Imaging (HSI)

“Dual-camera design for hyperspectral and panchromatic imaging, using a wedge shaped liquid crystal as a spectral multiplexer” | |)/S/URI LINK

“Discriminative Reconstruction for Hyperspectral Anomaly Detection With Spectral Learning” LINK

“Classification of common recyclable garbage based on hyperspectral imaging and deep learning” LINK

“Based on hyperspectral polarization to build the quantitative remote sensing model of jujube in Southern Xinjiang” LINK

“Study on quality distribution characteristics of jujube canopy based on multi-angle hyperspectral polarization” LINK

“Germination Prediction of Sugar Beet Seeds Based on HSI and SVM-RBF” LINK




Chemometrics and Machine Learning

“ATR-MIR spectroscopy to predict commercial milk major components: A comparison between a handheld and a benchtop instrument” LINK

“Simultaneous determination of goat milk adulteration with cow milk and their fat and protein contents using NIR spectroscopy and PLS algorithms” LINK

“Optimization and comparison of models for prediction of soluble solids content in apple by online Vis/NIR transmission coupled with diameter correction method” LINK

“Building kinetic models for apple crispness to determine the optimal freshness preservation time during shelf life based on spectroscopy” LINK

“Incorporation of two-dimensional correlation analysis into discriminant analysis as a potential tool for improving discrimination accuracy: Near-infrared spectroscopic discrimination of adulterated olive oils.” LINK

“Spectroscopic techniques combined with chemometrics for fast on-site characterization of suspected illegal antimicrobials” LINK




Environment NIR-Spectroscopy Application

“Improved mapping of soil heavy metals using a Vis-NIR spectroscopy index in an agricultural area of eastern China” LINK




Agriculture NIR-Spectroscopy Usage

“The Use of Multi-temporal Spectral Information to Improve the Classification of Agricultural Crops in Landscapes” LINK

“Remote Sensing, Vol. 12, Pages 1308: Machine Learning Based On-Line Prediction of Soil Organic Carbon after Removal of Soil Moisture Effect” LINK

“Detection of Nutrition and Toxic Elements in Pakistani Pepper Powders Using Laser Induced Breakdown Spectroscopy” LINK

“Agriculture, Vol. 10, Pages 177: Prediction of Soil Oxalate Phosphorus using Visible and Near-Infrared Spectroscopy in Natural and Cultivated System Soils of Madagascar” LINK




Forestry and Wood Industry NIR Usage

“Linear Discriminant Analysis of spectral measurements for discrimination between healthy and diseased trees of Olea europaea L. artificially infected by Fomitiporia …” LINK




Food & Feed Industry NIR Usage

“Quantification of Ash and Moisture in Wheat Flour by Raman Spectroscopy” LINK

“Visualization accuracy improvement of spectral quantitative analysis for meat adulteration using Gaussian distribution of regression coefficients in hyperspectral …” LINK




Laboratory and NIR-Spectroscopy

“Laboratory-Scale Preparation and Characterization of Dried Extract of Muirapuama (Ptychopetalumolacoides Benth) by Green Analytical Techniques” LINK




Other

“Machine vision detection of pests, diseases, and weeds: A review” LINK

“On-Site Identification of the Material Composition of PV Modules with Mobile Spectroscopic Devices” LINK

“Synthesis of N-Doped ZnO Nanocomposites for Sunlight Photocatalytic Degradation of Textile Dye Pollutants” LINK





.

NIR-Predictor – Manual


NIR-Predictor – Manual

Predicting Spectra

It’s easy to use with NIR-Predictor,
just drag & drop your data for getting the prediction results.

It supports an automatic file format detection.
So you don’t need to specify the instrument type and settings! See the list of supported formats and NIR Vendors: NIR-Predictor supported Spectral Data File Formats

Use the included data to checkout how it feels:

  1. Open the demo Spectra folder by using the Menu > Open Demo Spectra or press F8.
    There are files with spectra from different Vendors.

  2. Drag & drop a spectra file onto the NIR-Predictor window (or press Ctrl+O as for ’Open some files).

  3. The spectra will be

    • loaded
    • pre-processed
    • predicted and
    • reported

Note:
All the steps are fully automatic.
All calibrations that are compatible with the spectra, will produce prediction results in one go.
To select specific calibrations choose the Application. Where the ” ” empty means use all the calibrations.
To define a Application read more in chapter “Applications”

Hint:
To get access to Statistics of Predictions and Reports use the Menu > Show more/less (Ctrl+M) or you can simply resize the window. Here you can also re-do the Analyze step manually with changed inputs (e.g. Result Ordering).


Creating your own Calibrations

How it works – step by step

  1. You have measured your samples with you NIR-Instrument Software.
    And got the Lab-values of these samples.

    samples
    -> measured NIR-spectra
    -> Lab-references analytics

  2. Now you need to combine these data.

    NIR-spectra + Lab-references
    -> PropertiesBySamples

    Note: If you combined these data already in your NIR software used,
    and you can export it as a JCAMP-DX file then use
    Menu > Create Request File .req ... (F2)
    and read the “Help.html” and NIR-Predictor JCAMP.
    Else proceed as below.

    The NIR-Predictor provides tooling for that:

    Menu > Create Properties File... (F6)

    Select the folder with your NIR spectra measured for an application.
    NIR-Predictor creates a customized Properties file template for that data to enter the Lab values.

    Note: You don’t need to specify your instrument or vendor or an application. It’s all done automatically. And also the sample spectra are detected and grouped automatically!

  3. Use your favorite editor or spreadsheet program to enter and copy&paste
    the Lab-references Values into the columns “Prop1”, “Prop2” etc. and save the file.

  4. A final check of your entered data is done by NIR-Predictor,
    to make sure your data ist complete and all is fine.

    Menu > Create Calibration Request... (F7)

    Select the folder with the filled file.
    A CalibrationRequest.zip is created with the necessary data
    if enougth diverse Lab values are entered.

  5. Email the CalibrationRequest.zip file
    to info@CalibrationModel.com to develop the calibrations.

  6. When your calibrations are ready, you will receive an email with a link
    to the CalibrationModel WebShop where
    you can purchase and download the calibration files,
    that work with our free NIR-Predictor software without internet access.

    Note: Your sent NIR data is deleted after processing.
    We do not collect your NIR data!

Note: Further details can be found under “Create Properties File” and “Create Calibration Request”.


Applications

The Application concept allows to group multiple Calibrations together for a Application. By selecting an Application before prediction, only the Calibrations belonging to the Application will be used for Prediction. In the Demo Data this is used to have multiple spectrometer as Application. This can be used easily as e.g. as Application “Meat Products” containing Fat and Moisture Calibration.

To create an Application, create a folder with the Application’s name inside the Calibrations folder, and move/copy all the Calibrations files to this Application folder. To remove a Calibration from the Application, remove the Calibration file from the Application folder.

After creating an new Application folder, press menu Search and load Applications (F4) to update the NIR-Predictor dialog where the Application can be selected via the dropdown list. You don’t need to close the NIR-Predictor.

After moving Calibration files around, press menu Search and load Calibrations (F5) to update the NIR-Predictor dialog.

The use-all case

In the NIR-Predictor dialog where the Application can be selected via the dropdown list, the empty "" name means that all (yes all) valid Calibrations will be used for prediction.

Note: The Prediction Report will contain only results from spectral compatible Calibrations with the given spectra. That allows to automatically handle the multi vendor NIR instrument usage.


Prediction Result Report

Histograms of Prediction Values per Property

Shows the distribution of the predicted results per calibration. The histogram range contains the range of the calibrated property and includes the predicted results.

The histogram bar (bin) color is defined as follow:

  • blue : all predictions inside calibration range.
  • red : all predictions outside calibration range.
  • orange : some overlaps with calibration range.
    So not all spectra in a orange bin are outside calibration range.
Histograms

Note: Predicted values are always shown in Histogram table and Prediction Value List table, even if the spectrum does not fit into model (spectrum different to model, aka Residual Outlier) shown as Out = X.

Note: Old browsers like Microsoft Internet Explorer 11 don’t support the grafics for Histogram charts. Use an current browser like Firefox or Chrome or Edge.

Note: If your browser opens the report too slow, try to deactivate some browser plugins, because they can filter what you look at and some add-ons are really slow.

Spectra Plot Thumbnail on the Prediction Report

Visualizes the min,median,max spectrum of the spectra dropped as files on the NIR-Predictor. This gives a minimal and good spectral overview of the predicted property results.

  • Spectra Plot color legend: min,median,max spectrum by predicted property or if no calibration is available by spectral intensity.

  • The min,median,max is determined from the predicted properties or if not available from the intensity of the spectra.

  • Beside the histogram of the predicted properties, where the distribution can be seen, the spectra shown are the ones from min,median,max predicted property.

  • This gives a minimal and good spectral overview of the predicted property results.

  • The “Spectral Range” and number of datapoints is shown in the Prediction Report Header below the listed spectra files.

  • To zoom the spectra plot a little, zoom the report in the browser (hold ctrl + mouse wheel, or pinch on touch screen).

  • The spectra plots and histograms are stored with the report and can be archived.

Note

  • Note that the spectra are shown in the raw values that are loaded, they are not shown pre-processed as the calibration model uses them to make the prediction.

  • Note that the median property spectrum is the median from the predicted property pobulation and not the “median” of the calibration property range.

  • Note that in the multi calibration prediction case, the spectra are selected for each property based on the related predicted property values and so the spectra plots shows typical different spectra.

Spectra Plot

What is an Outlier? What are Outliers?

The spectrum is an outlier to the model, if the spectrum is not similar with the spectra and lab-values the model is built with.

This legend is shown on each NIR-Predictor prediction report below the results:

Outlier (Out) Symbol Description
X : spectrum does not fit into model (spectrum different to model)
O : spectrum is wide outside model center (spectrum similar to model but far away)
= : prediction is outside upper or lower range of model (property outside model range)
– : spectrum is incompatible to calibration

There are 3 outlier cases (X, O, =) and the incompatible data case “-“.
The bad case is “X”
the medium case is “O”
and soft case is “=”.

The technical names in literature correspond to:
X : Spectral Residual Outlier
O : Leverage Outlier
= : Property Range Outlier

These 3 outlier cases can appear in combinations, like “XO=” or “XO” or “O=” or “X=”.
The more outlier marker are shown the more likely the spectrum is an Outlier.

The default setting in NIR-Predictor Menu > “Report with Simplified Outlier Symbols”
is ON, that will show only the worst case instead of all combinations to have a simplified minimal information.
if OFF, that will show the combinations (e.g. “XO=” or “XO” or “O=” or “X=”), which is more informative for analyzing problem cases.


Some hints to avoid these Outliers:

X : spectrum does not fit into model (spectrum different to model)
Check if the spectrum is noise only, or has no proper signal.
That can happen when measured past the sample or measured into the air or at a different substance.
If you have multiple NIR instruments of the same type, use spectra measured with different instruments for the calibration.

O : spectrum is wide outside model center (spectrum similar to model but far away)
Sample temperature has an effect on NIR spectra shape, use spectra measured at different (typical use) temperatures (sample temperature, instrument temperature).

= : prediction is outside upper or lower range of model (property outside model range)
Use more spectra for the calibration in the Lab value region where your special interest is.
If the predicted value is only a little bit out of the calibration range, it can be Ok.
Add these spectra to the calibration spectra (with the Lab values), to extend the prediction range of the calibration.

– : spectrum is incompatible to calibration
The spectra (from the NIR instrument) has a different wavelength range or a different resolution than the spectra used for calibration.
Check Instrument settings (wavelength range, resolution)

Result Ordering

To change the ordering, a drop-down-box is located below the Analyze button. If there is an analysis from the current session, and the Result Ordering is changed, the data is re-Analyzed and reported with the new Result Ordering setting. That allows to compare the different orderings. The Result Ordering is listed in the Prediction Report above the Prediction Value List and stored in the settings.

The order/sorting of the prediction results of the spectra can be defined:

  • GivenOrder (default) the given order of the spectra from file select dialog or drag&drop

*) sorted : ascending sort

  • Date_Name sorted by Date (if any) and then by Name
  • Name_Date sorted by Name and then by Date
  • Date_NamesWithNumbers sorted by Date (if any) and then by Name with number logic
  • NamesWithNumbers_Date sorted by Name with number logic (e.g. “ABC1” is before “ABC002” ) and then by Date

*) as above but sorted Rev : reverse sort = descending sort

  • Rev_Date_Name
  • Rev_Name_Date
  • Rev_Date_NamesWithNumbers
  • Rev_NamesWithNumbers_Date

E.g. with reverse sort by Rev_Date_Name, the newest spectra appear on top.

Depending on how many calibrations are used the result table is getting broader. To print the report (e.g. to Adobe PDF, FreePDF or Microsoft XPS), sometimes the landscape format is shorter in number of pages or in portrait a scale of 80% fits nicely. Or try another internet browser (Mozilla Firefox, Google Chrome, Microsoft Edge, …) to print the report and set the browser as your default browser so it will be opened by default.

Archiving Reports

Each report is contained in one file only, including the grafics. To save storage space the report file folder can be compressed to a zip file (.zip, .7z).


Enter lab values to NIR spectra

Entering the laboratory reference values for NIR calibrations

We have developed specialized tools into NIR-Predictor to combine the NIR and Lab data is a sample-based safe manner.

The main target is to improve Data Quality during the step of combining of the Lab data and the NIR data, because to model a good reliable calibration the data that build the base needs to be of high quality.

It also simplifies to enter the lab values manually to the corresponding NIR data, because of automatically grouping repeated NIR measurements of the same sample, so the lab values can be entered sample based and not by spectrum.

It helps to avoid false reference data, because of the broken relation of NIR spectra and reference values, data entry on the wrong position in the table.

And Helps to detect errors of duplicated or multiple copies of spectra files, and checks for inconsistencies in Date-Time and Sample-Naming. It also checks for missing values.

That all increases the Data Quality for the next step of Calibration Development, and makes data entry a less time consuming and less risky work.

How it works

  1. Menu > Create Properties File... (F6) select the folder with your NIR spectra measured for an application. NIR-Predictor creates a Properties file template for that data : PropertiesBySamples.csv.txt

  2. Use your favorite editor or spreadsheet program to enter and copy&paste the Lab Values into the columns and save the file.

  3. Menu > Create Calibration Request... (F7) select the folder with the filled file for a last check and a Calibration Request file is created with the needed data as a single zip file.

  4. Email the Calibration Request file to info@CalibrationModel.com to develop the calibrations.

Ok that is it, the NIR-Predictor guides you through the steps needed. And if you need to know more details, the Chapter “Create Properties File” is for you.

Create Properties File

Note:

  • If you have (exported) JCAMP-DX files containing the Lab-Values, you don’t need to do this step.
    You can send the JCAMP file with your Request (.req) file directly to the calibration service at info@CalibrationModel.com.
  • If your JCAMP-DX files does NOT contain Lab-Values, this is a way to go.

For calibrating the spectra to the lab-values you need to assign the lab-values to the spectra. The easiest way is to have a table where each spectrum (row) is linked to multiple lab-values (columns). This function Create Properties File build such a table for the selected spectra folder automatically!

This table is stored in the file PropertiesBySamples.csv.txt. This can be created for any spectra folder you like. The file extension is .csv.txt to make it easy to edit in a text editor and also in a spreadsheet (excel). The columns are standard TAB separated.

The file header line contains :

Sample Replicates Names Prop1 Prop2 Prop3 DateFirst DateLast Hashes

Where Name and Date describes the spectrum.

Prop1, Prop2, Prop3 are the place to enter the Lab Reference Concentrations properties corresponding to each spectrum. It can be extended to Prop4, Prop5, … etc. Of course you can enter real word names like “Fat (%)” instead of “Prop1”. It’s recommended to put the measurement unit beside the name.

Replicates is the number of replicated or repeated spectra of a sample that is grouped together in the Sample based property file. Sample name and the DateFirst / DateLast between the sample spectra are measured.

Date format is ISO-8601. Missing Dates are 0002-02-02T00:00:00.0000000.

If the file PropertiesBySamples.csv.txt already exist in the selected folder, the user will be notified (it will not be overwritten, because the file may contain user entered Lab-values). The Lab Reference Concentrations values are initialized to 0 (zero) and needed to be changed.

Note: 0 is not interpreted as missing value! If you have a 0 concentration value, put in 0 or 0.0 .

The entry of properties is as easy as possible, because it’s organized by Sample (and not by Spectra), so it’s like your Lab-Value Table that is sample based. The sample rows are sorted in a special way by Sample name. Sorting by Date or alphabetically by Sample can done easily in a spreadsheet program.

Note: when coping lab values to the samples make sure they correspond, so that there are no gaps and the sorting is the same.

The Spectra (rows) are initially sorted by name (and date) to have the replicates/repeats together. You can sort for your convenience in a spreadsheet program.

Enter the Lab Reference Concentrations to the spectra/sample.

Enter the Lab-Values in spreadsheet (e.g. Excel) or a text editor (e.g. Notepad++). If done, use the next menu Create Calibration Request.

Hints: Data handling:

  • The NIR-Predictor creates the PropertiesBySamples.csv.txt once, after that the user is responsible for its content. That means NIR-Predictor does not change this file anymore.

  • You can remove entire rows (spectra) in the property file. You don’t need to remove the spectra files. The NIR-Predictor is aware of the relation, the PropertiesBySamples.csv.txt defines what will be calibrated.

  • How to add more spectra files?

    The additional spectra can be handled in a separate folder, create the property file and copy the spectra to the other folder and copy/merge the property files together in your editor or spreadsheet.

    Or

    Copy the spectra into the folder, rename the PropertiesBySamples.csv.txt to e.g. “PropertiesBySamples-Part1.csv.txt” and use Create Properties File to create a new PropertiesBySamples.csv.txt with all the spectra. You can copy/merge the content of the Properties files together in your editor or spreadsheet.

  • What happens with possible duplicate rows? It does no harm to the Calibration because we do an exact checking and data cleaning in the calibration process.

  • What happens to duplicate spectra names? The spectra names are not relevant for the calibration process. The spectra names are helpful to assign the lab-values to the corresponding spectrum entry. That’s why the table is initially sorted by name. The spectra names can be edited by the user.


Create Calibration Request

The menu function Create Calibration Request packs a created Properties file (see “Create Properties File”) and it’s linked spectra files in a compressed ZIP file for sending to the CalibrationModel.com Service.

Please note that the number of measured quantitative samples need to be at least 60 . That means you need at least 60 different spectra (not counting the replicate/repeated measurements).

It shows additional property information about the data you have entered, like – the property type (Quantitative) – it’s range (min – max) and – the number of unique values and – if the Lab-values are enough diverse to get calibrated.

First select the folder with the PropertiesBySamples.csv.txt and measured spectra files of samples you have Lab-values. The data is checked and you get notified what is missing or might be wrong. If something needs to be changed, edit the PropertiesBySamples.csv.txt and do Create Calibration Request again. Your last selected folder is remembered, so you can press return in the folder selection dialog.

Hint: The keyboard shortcuts for redoing it after you edited some entries is : F7 Return – that allows you to get the property information quickly.

Hint: If you open the PropertiesBySamples.csv.txt in a spreadsheet program, you can create Histogram plots of the entered Lab-values, to see in which range are to less samples measurements.

When all is fine

When all is fine the “CalibrationRequest.zip” file is created for that data.

The ZIP file contains:

  • your PropertiesBySamples.csv.txt
  • your personal REQuest file for your computer system, that looks like
    e.g. “337dcdc06b2d6dfb0b5c4bba578642312edf2ae84d909281624d7e26283e8b07 WIN-GB0PB48GSK4.req”
  • the spectra data files

Note: If the CalibrationRequest.zip file is already created and you change the PropertiesBySamples.csv.txt make sure to delete the old CalibrationRequest.zip file first! In the dialog it states if it was successfully created or NOT because it already exist. So you are always on the safe side.

Note: CalibrationRequest.zip file name contains the property names to know what would be calibrated and at the end an identification number for referencing the file. E.g. “CalibrationRequest ‘Prop1’ – ‘Prop2’ h31T3wOH.zip”


Program Settings

  • The users program settings are stored in UserSettings.json
  • The program counters are stored in GlobalCounters.json

Further References