Spectroscopy and Chemometrics News Weekly #47, 2015

Near Infrared

NIR-Sensor ermittelt Trockensubstanz während der Mischwagenbefüllung | Futterkomponente LINK

Ultra-low maintenance FTNIR analyzer for the refining & petrochemical industries | pauto LINK


Seeing Through Crude Oil for Efficient Oil Separations using Short-Wave Infrared (SWIR) Cameras – AZoSensors LINK


RoboBees Can Fly and Swim. What’s Next? Laser Vision – Smithsonian UAS UAV LINK


Scientists create an all-organic UV on-chip spectrometer – The U.S. Department of Energy’s Ames LINK


… detection of contaminants in agro-food products, … melamine levels in milk using vibrational spectroscopy LINK


Examining Pigmented Human Tissue using SWIR Raman Spectroscopy – AZoSensors LINK


SCiO Molecular Scanner UNBOXING – Video LINK


Dear NIR-Spectrometer vendors, this is about how you can improve customer web-traffic | NIRS Spectrometer LINK

Efficient development of new quantitative prediction equations for multivariate NIR spectra | spectra LINK

How to Develop Chemometric Near-Infrared Spectroscopy Calibrations in the 21st Century? | NIR LINK

How to Develop Near-Infrared Spectroscopy Application Today? | pharma lab analysis chemist TechTrends LINK

Improve chemical analysis accuracy by optimized chemometric models for Near-Infra-Red (NIR) Spectroscopy LINK

Improving Accuracy, Precision and Robustness of NIR-analysis LINK

NewsLetter: Spectroscopy and Chemometrics News Weekly 46, 2015 | Molecular Spectroscopy NIRS Chemometrics Raman LINK

Pro Tip: The NIR calibration is the central key to accurate NIR measurement LINK

Services for professional Development of Near-Infrared Spectroscopy Calibration Methods | NIR Quality Testing LINK

Procedures for NIR calibration – Creation of NIRS spectroscopy calibration curves

Do you know the effect that you prefer to try out their favorite data pretreatments in combination and often try the same wavelength selections based spectra of the visualized?

You try as six to ten combinations until one of them selects his favorite calibration model, to then continue to optimize. Since then suddenly fall to outliers, because it goes in depth, so is familiar with the data, we know now the spectra of numbers of outliers and is familiar with the extreme values.

Now, the focus is on the major components (principal components, Latent Variables, factors) and makes sure not to over-fit and under-fit not to. The whole takes a few hours and finally one is content with the model found.

So what would happen if you all in the beginning tried variants found outliers removed and re-evaluated and compared? The results would be better than that of the previous model choice? One does not try out? Because it is cumbersome and takes hours again?

We have developed a software which simplifies this so that also the number of model variations can be increased as desired. The variants generation is automated with an intelligent control system, as well as the optimization and comparing the models and finally the final selection of the best calibration model.

Our software includes all the usual known data pretreatment methods (data pre-processing) and can combine them useful. Since many Preteatments are directly dependent on the wavelength selection, such as the normalization the determined within a wavelength range of the scaling factors to normalize the spectra so that pretreatments with the wavelength ranges may be combined. So a variety of settings sensible model comes together that are all calculated and optimized. For the automatic selection of the relevant wavelength ranges, different methods are used, which are based on the spectral intensities. Thus, for example, regions with total absorption is not used, and often interfering water bands removed or retained.

Over all the calculated model variations as a summary outlier analysis can be made. Are there any new outliers (hidden outlier) discovered, all previous models can be automatically recalculated, optimized and compared without these outliers.

From this great number of calculated models with the statistical quality reviews (prediction performance) the optimum calibration can now be selected. For this purpose, not simply sorting by the prediction error (prediction error, SEP RMSEP) or the coefficient of determination (coefficient of determination r2), but by several statistical and test values are used jointly toward the final assessment of optimal calibration.

Thus we have created a platform that allows the highly automated work what a man can never do with a commercial software.

We therefore offer the largest number of matched to your application problem modeling calculations and choose the best calibration for you!

This means that our results are faster, more accurate, robust and objective basis (person independent) and quite easy for you to apply.

You have the full control of the models supplied by us, because we provide a clearly structured and detailed blueprint of the complete calibration, with all settings and parameters, with all necessary statistical characteristics and graphics.

Using this blueprint, you can adjust the quantitative calibration model itself in the software you use, understand and compare. You have everything under control form model creation, model validation and model refinement.

Your privacy is very important to us. The NIR data that you briefly provide us for the custom calibration development will remain of course your property. Your NIR data will be deleted after the job with us.

Interested, then do not hesitate to contact us.

Proof of Concept

Chemometric software competitions (aka shootouts) are a good way to check algorithms, software and knowledge against all other experts in the field.

Imagine that the prediction results can be produced with any kind of software and newest algorithms.

And we just use PLS right to generate models that can be used on all NIR software systems, because PLS is a quasi standard, supported in all major chemometrics software.

Our software framework reached very good results, got gold (rank #1) and silver (rank #2) during well known international NIR Chemometric software shootouts* so far, the competitions are held bi-annual.

Rank / competitors  Competition / Conference  Year
 #1 / 1  **  Kaji / ANSIG  2014
 #1 / 150  Kaji / ANSIG  2012
 #2 / ???  IDRC / IDRC  2012
The Kaji Competition

A set of NIR spectral data will be available for downloading from the ANISG website and contestants will be asked to find and explain a “best” chemometric model to robustly predict samples of the same type.
A panel will select the three “best” entries based on the predicted results and spectroscopic explanation of the products and attributes of interest. 


The IDRC Competition

The Software Shootout has been a staple of the IDRC. It is a competition amongst participants of the conference that aims at determining the person who developed the best model and obtained the lowest prediction error for a particular problem.
Every IDRC, a new challenge is proposed to participants. The challenge consists of a data set with calibration, test and a validation set.
Participants are given target values for the calibration and test sets but must do their best to develop a model that will predict the validation set as accurately and precisely as possible. Challenges from all sorts of fields of NIRS have been used (agriculture, biomedical, pharmaceutical, soil, …).


*) The author was unable to present the results at the conferences, so this ranking was not official but confirmed by the shootout organizers. Thanks go to Benoit Igne, IDRC 2012 shootout organizer and Steve Holroyd, Kaji Competition organizer at ANISG Conference 2012.  


Our chemometric software framework can significantly reduce the time spent for NIR method development and fine optimization. The time saving can be achieved through highly automated experiments and the usage of cloud computing. Calibrations are built and evaluated using automated good practices protocols resulting in useful, precise and robust Calibrations. The high number of experiments enables a deep screening of the solution domain to find the optimum calibration settings, something currently unavailable in standard chemometric software.

**) We were the only participator that got the 4 competition tasks (4-times more than usual) completed in that short time and submitted the fully documented results. After the competition, the information was given, that the data was originated from forages and the constituents were dry matter, organic matter digestibility, protein and ash. Thanks go to Daniel Cozzolino, Kaji 2014 Competition organizer.

Who uses this kind of NIR Calibration Service

  • NIR Laboratory Analyzer users
  • NIR Laboratories
  • NIR Analytical Laboratories
  • NIR Analytical Testing
  • NIR Analytical Services
  • NIR Laboratory Services
  • Contract Testing Laboratories
  • NIR Calibration Specialist
  • NIR Analysis Services
  • NIR Food Safety, food handling, nutrition, agriculture
  • NIR Feed and Forage
  • plant breeding companies
  • NIR Lab Instrument Services
  • NIR Quality Assurance & Quality Control (QA/QC)
  • Quality control laboratory, analytical laboratories
  • NIR Forensics
  • NIR Process Control
  • NIR Process Analytical Technology (PAT)
  • Research & Development, new analytical methods development
  • R&D Chemist, Modelers
  • Research Scientists, Scientist Analytical Development
  • NIR Spectroscopy PhD thesis
  • NIR Chemometric PhD thesis
  • NIR Application Specialist
  • Application Manager/Application Specialist
  • Analytical Method Development specialist
  • Product Specialist Spectroscopy
  • NIRS Application Developer
  • NIR Analytical Scientist
  • NIRS Method Developer
  • NIR feasibility studies
  • NIR sales force
  • NIR salesman
  • NIR vendors
  • NIR Users
  • NIR Chemometricians
  • NIR Spectroscopists
  • PAT Engineer
  • Quality Manager
  • Manager Analytical Testing
  • Analytical Development Engineer
  • Analytical Engineer
  • Lab Manager
  • etc.

Potential usage of NIR analysis and its industry fields of applications

What is typically measured by NIR Analysis? NIR Calibrations are used for the determination of the content of moisture, fat, protein, starch, lactose, fructose, glucose, alcohol, amino acids, oil, sugar, fiber, salt, Brix, caffeine, lysine, ash, gluten, etc. Where is near-infrared spectroscopy analysis used? NIR Analysis used in the industry area of
  • Agriculture
  • Food and Feed
  • Food and Beverages
  • Food and Dairy
  • Malt houses and breweries
  • Milling and Bakery
  • Flour, Grain milling and oils
  • Sugar
  • Feed Ingredients
  • Edible Oils
  • Meats
  • Animal feed
  • Aqua Feed
  • Pet Food and Animal Proteins
  • Dried and Wet Forage
  • Beverage and Biofuels
  • Chemical and Pharma
  • Pharmaceuticals
  • Pulp and Paper
  • Petro
  • Oil and Gas
  • Plastics
  • Polymers
  • Textiles
  • Packaging
  • Environmental
  • Forensics
  • Academia
  • Cosmetics
  • Health care

What is a NIR calibration used for?

NIR calibrations are used for NIR contents analysis as a productive analytical method. That is a two step procedure.
  1. A NIR analyzer does a non-destructive optical scan of a sample that yields a measured spectrum in seconds.
  2. A NIR calibration model can quantitatively predict (analyze, determine, estimate) multiple constituents, ingredients, contents, analytes, assay, API and other parameters and attributes (chemical, physical, biological, biochemical, sensory) summarized as properties, out of a single spectrum in milli seconds.
The NIR analysis is a very fast non-destructive analysis method that can replace or backup slower methods like wet chemical analysis, chemistry laboratory, sensory panels or rheology (viscosity). Or a NIR calibration can open the door to new possibilities of analytics, quality assurance and process control, by developing calibration models for parameters that seems to be impossible, because they are based on human knowledge, empirical values or sensory like taste value. If you have an NIR instrument, you can measure your samples systematically and thus develop your own calibration models.

Recent advanced chemometric methods

You are searching for recent advanced chemometric methods to get better calibration models for NIR? Methods and algorithms like:
  • Artificial Neural Networks (ANN)
  • General Regression Neural Networks (GR-NN)
  • RBF Neural Networks (RBF-NN)
  • Support Vector Machines (SVM)
  • Multiway Partial Least Squares (MPLS),
  • Orthogonal PLS (OPLS), (O-PLS), OPLS-AA, OPLS-ANN
  • Hierarchical Kernel Partial Least Squares (HKPLS)
  • Random Forest (RF)
  • etc.
and data pre-processing methods like
  • Extended Multiplicative Signal Correction (EMSC)
  • Orthogonal Signal Correction (OSC)
  • Dynamic Orthogonal Projection (DOP)
  • Error Removal by Orthogonal Subtraction (EROS)
  • External Parameter Orthogonalization (EPO)
  • etc.
that are partly available as modules for software packages like Matlab, Octave, R-Project, etc. Why invest a lot of time and money with new tools? Have you tried it really hard to optimize your calibrations with standard chemometrics methods like Partial Least Squares (PLS), Principal Component Regression (PCR) and Multiple Linear Regression (MLR) which are available in all chemometric software packages? Are you sure you have tried all the good rules and optimization possibilities? Get it done right with the compatible standard methods, we are specialized in optimization and development of NIR calibrations, let us help you, give us a try!

The Ghost Calibrator

To explain our service in an other way, I use an analogy between a book and a calibration. Building good calibrations is like writing a good book (a bestseller). You can write in a foreign language (chemometrics) with a high sophisticated word-processor (the chemometric software) that has a grammar checker (an outlier detection). Due to the complexity of the language (chemometrics) and the difficulty of the chosen book topic (the data) and the incomplete automatic grammar checker, you can never be sure if the grammar is correct and may not lead to misunderstanding (bad prediction performance). So the best way is to let a native language speaker check and correct the text. In that way (the analogy), you can see us even as a ghostwriter (a ghost calibration developer, a ghostcalibrator) that helps you, writing the book (with long year experience, consolidated knowledge, time saving, a lot of benefit). The analogy fits very well, because you can define the topic of the book (with your data). Finally you own the calibration and you have the full insight in how it is done. You have it under full control.

Optimize existing NIR calibrations

Compare the current model performance with the newly optimized NIR calibration. You want to measure your calibration talents with others, on your own data. Learn to do this if you use other improved calibration settings on your data. See what you can get more out of your data.


The NIR Calibration service offers the following benefit: Saving money
  • Improving the accuracy and reliability of already used NIR calibration models have high potential in various manufacturing processes as well as in quality assurance.
  • increased accuracy of analysis => better control of the production process => optimum process flow => better quality => less waste => more throughput.
  • quick and inexpensive to create professional NIR calibration models.
  • relief of their own staff
Time savings
  • for data cleaning (increasing data quality) – missing data, outlier search, wrong data (conflicting information), outlier removal
  • for the search for the optimal NIR model parameter settings (calibration set, wavelength selection, data pretreatments, factor selection)
  • for the calculation of different variations of the model
  • for the validation, evaluation and selection of the optimal model (error, SEP, RMSEP, RMSEC, RPD, fit, R2, bias, slope, …)
  • time-consuming calculation of huge calibration models
  • no long trial and error and waiting in the used NIR software until the calibrations seems to work
NIR analytical accuracy
  • higher reliability due to accuracy and robustness of NIR calibration models
  • the possibility of comparison with your own created or already existing or purchased NIR calibrations
  • what performance increase of analytical accuracy is possible
  • improvement of robustness with respect to change of the product matrix and possible instruments drift
Professional NIR calibration models
  • decades of experience in chemometrics for NIR spectroscopy
  • based on theoretical and applied good practice and know-how
  • application of various guidelines and rules
  • application of vendor-independent NIR chemometric software
  • outsourcing of NIR calibration method development and calibration equation maintenance
  • improving the robustness of NIR prediction model
  • avoid traps and pitfalls of the complicated chemometrics
Detailed results
  • The service provides optimal calibration settings for your NIR data.
  • You get full insight into the NIR calibration, as it is produced and detailed statistical values as a performance index assisted with graphics.