Spectroscopy and Chemometrics News Weekly #13-15, 2017

Chemometrics

NIR spectroscopy and cellulose content predicted coating build-up on drug layered pellets AAPSPT | h… LINK

Prediction of Soil Physical & Chemical Properties by Visible & Near-Infrared Diffuse Reflectance Spectroscopy in … LINK

How does multivariate calibration work for Raman monitoring? Bioprocess | LINK

Characterization of Mammalian Cell Culture Raw Materials by Combining Spectroscopy and Chemometrics. LINK

Near Infrared Spectroscopy Predicts Compositional & Mechanical Properties of Hyaluronic Acid-Based Engineered Cart. LINK

PAT for Continuous API Manufacturing Progresses – Chemometrics are applied to collected spectra to maximize the … LINK

Quantification of Lycopene,Carotene,Soluble Solids in Red-Flesh Watermelon Using On-Line Near-Infrared Spectroscopy LINK



Near Infrared

PAT-Based Control of Fluid Bed Coating Process Using NIR Spectroscopy to Monitor the Cellulose Coating on Pellets LINK

RISI Pulp: Fitnir Analyzers to supply FT-NIR online analyzer system to Harmac Pacific’s NBSK pulp mill in Nanaimo,… ht… LINK!

Paperindex Times: Harmac Pacific Selects Fitnir Analyzers To Supply Online Ft-Nir Analyzer LINK

Using Spectroscopy to Grade and Sort Fruit – choosing appropriate wavelengths – monitoring the entire NIR spectra LINK

Global Near Infrared Spectroscopy Market 2017 – Market Research News by | (press release) LINK



Infrared

Active Mode Remote Infrared Spectroscopy Detection of TNT and PETN on Aluminum Substrates LINK



Food & Feed

Congratulations to the winners of the foodscanner HorizonPrize! ScioScan cebit17 LINK!



Agriculture

From Crop Science to Space Exploration, Optical Sensing on the Rise | OpticalSensing LINK



Laboratory

Der Laborausrüster Sartorius kauft den Datenspezialisten Umetrics. Datenanalyse LINK



Petro

Visible and Near-IR Sensing: Plastic-optical-fiber-based ethanol sensor is simple, low-cost | NearIR LINK



Raman

Fructose and Pectin Detection in Fruit-Based Food Products by Surface-Enhanced Raman Spectroscopy (SERS) LINK



Other

A Retiree Discovers an Elusive Math Proof-And Nobody Notices – WIRED LINK



Spectroscopy and Chemometrics News Weekly #48+49, 2016

Get the Chemometrics and Spectroscopy News in real time on Twitter @ CalibModel and follow us.



Near Infrared

Cannabis Analysis – On-Site Determination of Cannabis Strength using FT-IR Spectrosocopy FTNIR Ingredients LINK

Near Infrared NIRS, GC and HPLC Applications in Cannabis Testing THC CBD LINK

Raman

What happens when you use Raman spectroscopy to discriminate between brands of extra-virgin olive oil LINK

Raman spectroscopy of chocolate bloom LINK


Hyperspectral

Hyperspectral photoluminescence imaging of defects in solar cells | solar cells via LINK


Agriculture

Soy meal Protein bands LINK

Vitamin C distribution in acerola fruit by near infrared hyperspectral imaging HSI LINK


Equipment

Spectroscopists need freedom to analyse their spectral data, uncoupled from spectrometer hardware! LINK!


Chemometrics

Quality parameters in Castanhola fruit by NIRS to development of prediction models using PLS … in laboratory scale LINK

Monitoring Process-Water Quality Using NIRS and PLSR with Prediction Uncertainty Estimation LINK


Food & Feed

NIR diffuse reflection analysis of fruit – Food Science & Technology LINK


Agriculture

Innovation für die Obstwirtschaft: Neue Ansätze zur Messung und Vorhersage der Apfelqualität MONALISA LINK


Other

Hackers beware! Faking 3D-printed products just got harder. Full-spectrum spectroscopy for the win! LINK!

3D NDVI, using a low cost multi spectral camera. LINK

On the Generation of Random Multivariate Data | Multivariate Data LINK


CalibrationModel.com

Spectroscopy and Chemometrics News Weekly 46+47, 2016 | Spectroscopy NIRS Multivariate DataAnalysis Software LINK

Spektroskopie und Chemometrie Neuigkeiten Wöchentlich 46+47, 2016 | NIRS Spektroskopie Multivariate DatenAnalyse LINK

Spettroscopia e Chemiometria Weekly News 46+47, 2016 | NIRS Spettroscopia Chemiometria LINK


Procedures for NIR calibration – Creation of NIRS spectroscopy calibration curves

Do you know the effect that you prefer to try out their favorite data pretreatments in combination and often try the same wavelength selections based spectra of the visualized?

You try as six to ten combinations until one of them selects his favorite calibration model, to then continue to optimize. Since then suddenly fall to outliers, because it goes in depth, so is familiar with the data, we know now the spectra of numbers of outliers and is familiar with the extreme values.

Now, the focus is on the major components (principal components, Latent Variables, factors) and makes sure not to over-fit and under-fit not to. The whole takes a few hours and finally one is content with the model found.

So what would happen if you all in the beginning tried variants found outliers removed and re-evaluated and compared? The results would be better than that of the previous model choice? One does not try out? Because it is cumbersome and takes hours again?

We have developed a software which simplifies this so that also the number of model variations can be increased as desired. The variants generation is automated with an intelligent control system, as well as the optimization and comparing the models and finally the final selection of the best calibration model.

Our software includes all the usual known data pretreatment methods (data pre-processing) and can combine them useful. Since many Preteatments are directly dependent on the wavelength selection, such as the normalization the determined within a wavelength range of the scaling factors to normalize the spectra so that pretreatments with the wavelength ranges may be combined. So a variety of settings sensible model comes together that are all calculated and optimized. For the automatic selection of the relevant wavelength ranges, different methods are used, which are based on the spectral intensities. Thus, for example, regions with total absorption is not used, and often interfering water bands removed or retained.

Over all the calculated model variations as a summary outlier analysis can be made. Are there any new outliers (hidden outlier) discovered, all previous models can be automatically recalculated, optimized and compared without these outliers.

From this great number of calculated models with the statistical quality reviews (prediction performance) the optimum calibration can now be selected. For this purpose, not simply sorting by the prediction error (prediction error, SEP RMSEP) or the coefficient of determination (coefficient of determination r2), but by several statistical and test values are used jointly toward the final assessment of optimal calibration.

Thus we have created a platform that allows the highly automated work what a man can never do with a commercial software.

We therefore offer the largest number of matched to your application problem modeling calculations and choose the best calibration for you!

This means that our results are faster, more accurate, robust and objective basis (person independent) and quite easy for you to apply.

You have the full control of the models supplied by us, because we provide a clearly structured and detailed blueprint of the complete calibration, with all settings and parameters, with all necessary statistical characteristics and graphics.

Using this blueprint, you can adjust the quantitative calibration model itself in the software you use, understand and compare. You have everything under control form model creation, model validation and model refinement.

Your privacy is very important to us. The NIR data that you briefly provide us for the custom calibration development will remain of course your property. Your NIR data will be deleted after the job with us.

Interested, then do not hesitate to contact us.

NIR Spectroscopy Calibration Report for quantitative predictive models

When you send your quantitative NIR spectra data to our NIR Calibration Model Service, you get a detailed calibration report (calibration protocol) of the found optimal calibration settings, so you are able to see all insights and easily re-build the model in your NIR/Chemometric software.

Here is a part of our calibration report, that exactly describes the data used in the calibration set (CSet), the validation set (VSet) and the test set (TSet). The numbers are the number ids of the spectra in your delivered NIR data file.


The calibration method settings and parameters are
Waveselection : the variable selection or wavenumber selection or wavelength selection
Pretreatments : the spectral data pre-processing
PCs : the number of  Principal Components (PC) or Latent Variables (LV)
Method : the modeling method algorithm used, e.g. PLS

Then the statistical analysis of the PLS model by the different sets (CSet, VSet, Tset).

Calibration Report

Statistical analysis of calibration, validation and test results : 1 Name, 2 Unit, 3 N : number of spectra, 4 N : number of samples, 5 Average spectra count per sample, 6 Reference values, 7 Min, 8 Mean, 9 Median, 10 Max, 11 Standard deviation, 12 Skewness : left (-) or right (+) lack of symmetry, 13 Kurtosis : flat (-) or peaked (+) shape, 14 Model statistics, 15 RPD, 16 R², 17 RMSEC, RMSEP, RMSET : root mean square of prediction errors, 18 SEC, SEP, SET : standard error (bias corrected), 19 Bias, 20 Skewness of prediction errors, 21 Kurtosis of prediction errors, 22 Intercept, 23 Slope, 24 Intercept (reverse), 25 Slope (reverse), 26 Sample Prediction Repeatability Error, 27 Sample Prediction Repeatability Error (of Missing data MSet)

This shows how we deliver the optimal settings. With the statistical values, the NIR model predicted values of all spectra and additional plots you are able to compare with your re-built model to verify that the models perform nearly equally.

Proof of Concept

Chemometric software competitions (aka shootouts) are a good way to check algorithms, software and knowledge against all other experts in the field.

Imagine that the prediction results can be produced with any kind of software and newest algorithms.

And we just use PLS right to generate models that can be used on all NIR software systems, because PLS is a quasi standard, supported in all major chemometrics software.

Our software framework reached very good results, got gold (rank #1) and silver (rank #2) during well known international NIR Chemometric software shootouts* so far, the competitions are held bi-annual.

Rank / competitors  Competition / Conference  Year
 #1 / 1  **  Kaji / ANSIG  2014
 #1 / 150  Kaji / ANSIG  2012
 #2 / ???  IDRC / IDRC  2012
The Kaji Competition

A set of NIR spectral data will be available for downloading from the ANISG website and contestants will be asked to find and explain a “best” chemometric model to robustly predict samples of the same type.
A panel will select the three “best” entries based on the predicted results and spectroscopic explanation of the products and attributes of interest. 

http://www.anisg.com.au/the-kaji-competition


The IDRC Competition

The Software Shootout has been a staple of the IDRC. It is a competition amongst participants of the conference that aims at determining the person who developed the best model and obtained the lowest prediction error for a particular problem.
Every IDRC, a new challenge is proposed to participants. The challenge consists of a data set with calibration, test and a validation set.
Participants are given target values for the calibration and test sets but must do their best to develop a model that will predict the validation set as accurately and precisely as possible. Challenges from all sorts of fields of NIRS have been used (agriculture, biomedical, pharmaceutical, soil, …).

IDRC


*) The author was unable to present the results at the conferences, so this ranking was not official but confirmed by the shootout organizers. Thanks go to Benoit Igne, IDRC 2012 shootout organizer and Steve Holroyd, Kaji Competition organizer at ANISG Conference 2012.  

Conclusion

Our chemometric software framework can significantly reduce the time spent for NIR method development and fine optimization. The time saving can be achieved through highly automated experiments and the usage of cloud computing. Calibrations are built and evaluated using automated good practices protocols resulting in useful, precise and robust Calibrations. The high number of experiments enables a deep screening of the solution domain to find the optimum calibration settings, something currently unavailable in standard chemometric software.

**) We were the only participator that got the 4 competition tasks (4-times more than usual) completed in that short time and submitted the fully documented results. After the competition, the information was given, that the data was originated from forages and the constituents were dry matter, organic matter digestibility, protein and ash. Thanks go to Daniel Cozzolino, Kaji 2014 Competition organizer.

Summary of the NIR Chemometric survey polls

Summary of the NIR Chemometric survey polls (as of end of Sept. 2013)

The interesting finding is that most of the answers fit the following pattern. The most companies that use NIR have one NIR Instrument and only one employee that is able to develop NIR calibrations. For that the most common off-the-shelf chemometrics program is used and spent 2 hours or over a month and therefore gets no calibration training about the complex topics like Chemometrics and NIR Spectroscopy or only once (introduction). The calibration maintenance ranges from never to 3 times a year. Interestingly, there was no one who uses portable NIR instruments. We continue our surveys, for the discovery of new trends. Conclusion Seeing this picture, we think that there is huge potential to improve the calibrations. Advanced knowledge can help individuals to build the calibrations with best practices and improve their models accuracy and reliability. Once the decision and investment in NIR technology is done, you should get the best out of your data, because this extra NIR performance can be given by calibration optimization. We offer this as an easy to use and independent service.

NIR Calibration Modeling

The majority of NIR calibrations are generated using a small number of different parameter settings and all too often are restricted to the time a user has available, their spectroscopic and chemometric knowledge and their ability (tedious use of the software) to choose and combine all the possible parameter settings required for good calibrations.

There are many published standards and guidelines (protocols) available for developing NIR calibrations from Standards Consortium such as ASTM, EMEA, ICH, IUPAC, ISO, USP, PASG etc. as well as many good recommendations and guidelines found in various textbooks and papers.

The difficulty with so many ‘Protocols’ for the NIR user is to have them all available and in their thought processes during calibration work and in addition to execute, check and challenge all calibrations generated manually. This is time consuming and sometimes boring repetitive work.

To simplify this for the person generating the NIR Calibrations, we have collected the good practices protocols and integrated them into our service that automates the calibration building and evaluation procedures.

to part 2

Benefit

The NIR Calibration service offers the following benefit: Saving money
  • Improving the accuracy and reliability of already used NIR calibration models have high potential in various manufacturing processes as well as in quality assurance.
  • increased accuracy of analysis => better control of the production process => optimum process flow => better quality => less waste => more throughput.
  • quick and inexpensive to create professional NIR calibration models.
  • relief of their own staff
Time savings
  • for data cleaning (increasing data quality) – missing data, outlier search, wrong data (conflicting information), outlier removal
  • for the search for the optimal NIR model parameter settings (calibration set, wavelength selection, data pretreatments, factor selection)
  • for the calculation of different variations of the model
  • for the validation, evaluation and selection of the optimal model (error, SEP, RMSEP, RMSEC, RPD, fit, R2, bias, slope, …)
  • time-consuming calculation of huge calibration models
  • no long trial and error and waiting in the used NIR software until the calibrations seems to work
NIR analytical accuracy
  • higher reliability due to accuracy and robustness of NIR calibration models
  • the possibility of comparison with your own created or already existing or purchased NIR calibrations
  • what performance increase of analytical accuracy is possible
  • improvement of robustness with respect to change of the product matrix and possible instruments drift
Professional NIR calibration models
  • decades of experience in chemometrics for NIR spectroscopy
  • based on theoretical and applied good practice and know-how
  • application of various guidelines and rules
  • application of vendor-independent NIR chemometric software
  • outsourcing of NIR calibration method development and calibration equation maintenance
  • improving the robustness of NIR prediction model
  • avoid traps and pitfalls of the complicated chemometrics
Detailed results
  • The service provides optimal calibration settings for your NIR data.
  • You get full insight into the NIR calibration, as it is produced and detailed statistical values as a performance index assisted with graphics.