Spectroscopy and Chemometrics News Weekly #48+49, 2016

Get the Chemometrics and Spectroscopy News in real time on Twitter @ CalibModel and follow us.

Near Infrared

Cannabis Analysis – On-Site Determination of Cannabis Strength using FT-IR Spectrosocopy FTNIR Ingredients LINK

Near Infrared NIRS, GC and HPLC Applications in Cannabis Testing THC CBD LINK


What happens when you use Raman spectroscopy to discriminate between brands of extra-virgin olive oil LINK

Raman spectroscopy of chocolate bloom LINK


Hyperspectral photoluminescence imaging of defects in solar cells | solar cells via LINK


Soy meal Protein bands LINK

Vitamin C distribution in acerola fruit by near infrared hyperspectral imaging HSI LINK


Spectroscopists need freedom to analyse their spectral data, uncoupled from spectrometer hardware! LINK!


Quality parameters in Castanhola fruit by NIRS to development of prediction models using PLS … in laboratory scale LINK

Monitoring Process-Water Quality Using NIRS and PLSR with Prediction Uncertainty Estimation LINK

Food & Feed

NIR diffuse reflection analysis of fruit – Food Science & Technology LINK


Innovation für die Obstwirtschaft: Neue Ansätze zur Messung und Vorhersage der Apfelqualität MONALISA LINK


Hackers beware! Faking 3D-printed products just got harder. Full-spectrum spectroscopy for the win! LINK!

3D NDVI, using a low cost multi spectral camera. LINK

On the Generation of Random Multivariate Data | Multivariate Data LINK


Spectroscopy and Chemometrics News Weekly 46+47, 2016 | Spectroscopy NIRS Multivariate DataAnalysis Software LINK

Spektroskopie und Chemometrie Neuigkeiten Wöchentlich 46+47, 2016 | NIRS Spektroskopie Multivariate DatenAnalyse LINK

Spettroscopia e Chemiometria Weekly News 46+47, 2016 | NIRS Spettroscopia Chemiometria LINK

Spectroscopy and Chemometrics News Weekly #2, 2016


“Measurement of aspartic acid in oilseed rape leaves using near infrared spectroscopy and chemometrics” LINK

Relationships betwn. volatile compounds & sensory characteristics in virgin olive oil by analytical & chemometric LINK

Near Infrared

“Quo Vadis Regulated NIR Analytical Procedures?” | NIRS LINK

“Rapid P test helps to hit a moving target in feed formulation” – Phytate-P & Total-P, phytase – NIR analysis LINK

An Ensemble Successive Project Algorithm for Liquor Detection Using Near Infrared Sensor | NIR NIRS LINK


Can technology help tackle the world’s waste crisis? – BBC News – near infrared sorting technology LINK


EAGLE Raman-S – Ibsen launches 800–1100 nm high throughput spectrometer for Raman spectroscopy LINK

Thermo Fisher Scientific to Acquire Affymetrix for $1.3b LINK

Process Control

Rising Demand for Quality Checks Paves Path for Global Process Spectroscopy Market – Industry Today LINK


Packaging Innovations, Label&Print & Empack 2016 – Events | Packaging Print LINK

Food & Feed

“Research and Markets: Global Food Safety Testing Market Worth USD 16.2 Billion by 2020″ LINK


Counterfeit pharmaceuticals are a $200 bn-a-year industry, & growing | IllicitTrade LINK


“MKS is Now MKS Data Analytics Solutions” | DataAnalytics LINK

“Using spidersilk to detect molecules” | fiber optics LINK

How to develop near-infrared spectroscopy calibrations in the 21st Century?

The Problem

Calibration modeling is a complex and very important part of NIR spectroscopy, especially for quantitative analysis. If the model is badly designed the best instrument precision and highest data quality does not help getting good and robust measurement results. And NIR Spectroscopy requires periodically recalibration and validation.

How are NIR models built today?

In a typical usage in industry, a single person is responsible to develop the models (see survey). He or she uses a Chemometric software that has a click-and-wait working process to adjust all the possible settings for the used algorithms in dialogs and wait for calculations and graphics and then to think about the next modeling steps and the time is limited to do so. Do we expect to find the best use-able or optimal model that way? How to develop near-infrared spectroscopy calibrations in the 21st Century?

Our Solution

Why not put all the knowledge a good model builder is using into software and let the machines do the possibilities of calculations and presenting the result? Designing the software that way, that the domain knowledge is built-in, not just only the algorithms for machine learning and make it possible to scale the calculations to multi-core computers and up to cloud servers. Extend the Chemometric Software with the Domain Knowledge and make as much computer power available as needed.

As it was since the beginning

User  → Chemometric Software → one Computer → some results to choose from

==> User’s time needed to click-and-wait for creating results

Our Solution

User → (Domain Knowledge → automatized Chemometric Software) → many Computers → the best models

==> User’s time used to study the best models and reasoning about his product / process

Note that the “Domain Knowledge” here does perfectly support the User’s product and process knowledge to get the things done right and efficient.

Scaling at three layers

  • Knowledge : use the domain knowledge to drive the Chemometric Software
  • Chemometric Software : support many machine learning algorithms and data pre-processings and make it automatic
  • Computer : support multi-core calculations and scale it to the cloud

The hard part in doing this, is of course the aggregation of the needed domain knowledge and transform it into software. The Domain Knowledge for building Chemometric NIR Spectroscopic models is well known and it’s huge and spreads multiple disciplines. Knowledge-driven software for computing helps to find the gold needle in the haystacks. It’s all about scaling that makes it possible. See Proof of Concept.

New possibilities

  • NIR users can get help working more efficient and getting better models.
  • New types of applications for NIR can be discovered.
  • Evaluation of NIR Applications to replace conventional analytical methods.
  • Hopeless calibrations development efforts can be re-started.
  • Higher model accuracy and robustness can be delivered.
  • Automate the experimental data part of your application study.
  • Person independent optimization will show new solutions, because it’s not limited by a single mindset => combining all the aggregated knowledge and its combinations.
  • Software independent optimization will show new solutions, because none of vendor specific limitations and missing algorithms are present => combining all open available algorithms and there permutations.
  • Computing service is included.

Contact us for trial

Your NIR data is modeled by thousands of different useful calibration models and you get the best of them! That was not possible before in such a easy and fast way! See How it works

Proof of Concept

Chemometric software competitions (aka shootouts) are a good way to check algorithms, software and knowledge against all other experts in the field.

Imagine that the prediction results can be produced with any kind of software and newest algorithms.

And we just use PLS right to generate models that can be used on all NIR software systems, because PLS is a quasi standard, supported in all major chemometrics software.

Our software framework reached very good results, got gold (rank #1) and silver (rank #2) during well known international NIR Chemometric software shootouts* so far, the competitions are held bi-annual.

Rank / competitors  Competition / Conference  Year
 #1 / 1  **  Kaji / ANSIG  2014
 #1 / 150  Kaji / ANSIG  2012
 #2 / ???  IDRC / IDRC  2012
The Kaji Competition

A set of NIR spectral data will be available for downloading from the ANISG website and contestants will be asked to find and explain a “best” chemometric model to robustly predict samples of the same type.
A panel will select the three “best” entries based on the predicted results and spectroscopic explanation of the products and attributes of interest. 


The IDRC Competition

The Software Shootout has been a staple of the IDRC. It is a competition amongst participants of the conference that aims at determining the person who developed the best model and obtained the lowest prediction error for a particular problem.
Every IDRC, a new challenge is proposed to participants. The challenge consists of a data set with calibration, test and a validation set.
Participants are given target values for the calibration and test sets but must do their best to develop a model that will predict the validation set as accurately and precisely as possible. Challenges from all sorts of fields of NIRS have been used (agriculture, biomedical, pharmaceutical, soil, …).


*) The author was unable to present the results at the conferences, so this ranking was not official but confirmed by the shootout organizers. Thanks go to Benoit Igne, IDRC 2012 shootout organizer and Steve Holroyd, Kaji Competition organizer at ANISG Conference 2012.  


Our chemometric software framework can significantly reduce the time spent for NIR method development and fine optimization. The time saving can be achieved through highly automated experiments and the usage of cloud computing. Calibrations are built and evaluated using automated good practices protocols resulting in useful, precise and robust Calibrations. The high number of experiments enables a deep screening of the solution domain to find the optimum calibration settings, something currently unavailable in standard chemometric software.

**) We were the only participator that got the 4 competition tasks (4-times more than usual) completed in that short time and submitted the fully documented results. After the competition, the information was given, that the data was originated from forages and the constituents were dry matter, organic matter digestibility, protein and ash. Thanks go to Daniel Cozzolino, Kaji 2014 Competition organizer.

NIR Calibration Modeling (Part 2)

( to part 1 )

All the below categories are implemented by using multiple different algorithms and formulas which leads to many different calibrations.

Steps in modeling
  • Data Cleaning – (bad data, missing values, duplicate elimination, spectral quality / intensity / noise, input value typing errors, …)
  • Initial Calibration set up – selection of calibration, validation and test samples
  • Wavelengths selection
  • Data preprocessing, pretreatments
  • Method calculation
  • Choosing the number of Principal Components / Latent Variables
  • Validation of calibration model / Statistics of performance – (accuracy, precision, linearity, repeatability, range, distribution, robustness / stability, sensitivity, simplicity, etc.)
  • Outlier examination and removal

The problem of choosing the optimal number of factors to find the optimum between underfitting and overfitting is solved by having multiple methods and protocols implemented leading to multiple calibrations.

The evaluation and the selection of the best calibration is based on many individual statistical values including the most popular RMSEP, SEP, Bias, SEC, R2 and PCs etc.

Results and Reporting

A detailed calibration report is provided detailing the best available calibration containing all calibration parameter settings and statistics of prediction performance of the calibration set, the validation set and the test set. A visual expression of the calibration is provided with the most importance plots.

Our service works with any quantitative NIR spectra data set in the standard JCAMP-DX format and uses mainly PLS and PCR to be compatible with other chemometric calibration software.

Summary of the NIR Chemometric survey polls

Summary of the NIR Chemometric survey polls (as of end of Sept. 2013)

The interesting finding is that most of the answers fit the following pattern. The most companies that use NIR have one NIR Instrument and only one employee that is able to develop NIR calibrations. For that the most common off-the-shelf chemometrics program is used and spent 2 hours or over a month and therefore gets no calibration training about the complex topics like Chemometrics and NIR Spectroscopy or only once (introduction). The calibration maintenance ranges from never to 3 times a year. Interestingly, there was no one who uses portable NIR instruments. We continue our surveys, for the discovery of new trends. Conclusion Seeing this picture, we think that there is huge potential to improve the calibrations. Advanced knowledge can help individuals to build the calibrations with best practices and improve their models accuracy and reliability. Once the decision and investment in NIR technology is done, you should get the best out of your data, because this extra NIR performance can be given by calibration optimization. We offer this as an easy to use and independent service.

Customized NIR Calibrations

Increase Your Profit with optimized NIR Accuracy

We help you to find the optimal settings for higher NIR accuracy and reliability.

You can build your own custom NIR calibration model with this valuable settings.

We offer a quantitative NIR Calibration development and optimization service.

New: White Paper about the details, what’s behind.

Improve NIR Measurement Accuracy

  • going closer to your product specification limits and maximize profitability
  • optimizing your models yield to process optimization and optimizing productivity
  • compete against other NIR vendors in a feasibility study (NIR salesman)

Easy to use

  • compatible with any NIR vendor
  • no installation, no learning
  • quantitative NIR Calibration Development as a Service


  • help users avoid common pitfalls of method development
  • before you validate and approve your solution for use in production process:
    • check if a better calibration can be found,
    • compare your calibration with other experts solutions.


  • no cumbersome trial-and-error modeling steps
  • calculation time is spent on our high performance infrastructure
  • fast results, developed calibrations within days

Fix price

  • fix costs, depends only on data size (not hourly rate for service)
  • huge saving in method development costs
  • easy to plan
More benefits, for whom and where, learn more , contact

NIR Calibration Modeling

The majority of NIR calibrations are generated using a small number of different parameter settings and all too often are restricted to the time a user has available, their spectroscopic and chemometric knowledge and their ability (tedious use of the software) to choose and combine all the possible parameter settings required for good calibrations.

There are many published standards and guidelines (protocols) available for developing NIR calibrations from Standards Consortium such as ASTM, EMEA, ICH, IUPAC, ISO, USP, PASG etc. as well as many good recommendations and guidelines found in various textbooks and papers.

The difficulty with so many ‘Protocols’ for the NIR user is to have them all available and in their thought processes during calibration work and in addition to execute, check and challenge all calibrations generated manually. This is time consuming and sometimes boring repetitive work.

To simplify this for the person generating the NIR Calibrations, we have collected the good practices protocols and integrated them into our service that automates the calibration building and evaluation procedures.

to part 2

NIR Spectroscopy and Chemometric surveys, polls and assessments (Part 2)

5. Calibration Precision
What do you believe, can NIR calibration models be more precise than reference values?

6. Calibration Maintenance
How often do you update your quantitative calibrations per year?

7. Quantitative Calibrations
How many quantitative (%) calibrations do you have in use?

8. Quantitative Parameters
In all your quantitative calibrations, how many parameters (properties) you have in total?

9. Qualitative Calibrations
How many qualitative (identification) calibrations do you have in use?

Please vote and see the assessments below.
Part 1, Part 3
Calibration Precision
What do you believe, can NIR calibration models be more accurate than reference method?
Calibration Maintenance
How often do you update your quantitative calibrations per year?
Quantitative Calibrations
How many quantitative (%) calibrations do you have in use?
Quantitative Parameters
In all your quantitative calibrations, how many parameters (properties) you have in total?
Qualitative Calibrations
How many qualitative (identification) calibrations do you have in use?

Part 1, Part 3

NIR Spectroscopy and Chemometric surveys, polls and assessments

1. Calibration Developers
How many persons in your company are able to develop a NIR Calibration?

2. Calibration Development
How much time do you spend to develop a calibration model?

3. Chemometric Software / Spectroscopy Software
Which Chemometric Software are you using for NIR?

4. NIR Spectrometer Brand
Which NIR Spectrometer Brand do you use?

Please vote and see the assessments below.

Part 2, Part 3
Calibration Developers
How many persons in your company are able to develop a NIR Calibration?
Calibration Development
How much time do you spend to develop a calibration model?
Chemometric Software
Which Chemometric Software are you using for NIR?
NIR Spectrometer Brand
Which NIR Spectrometer Brand do you use?

Part 2, Part 3