Spectroscopy and Chemometrics News Weekly #31, 2018

Chemometrics

How to Configure the Number of Layers and Nodes in NeuralNetworks: BigData DataScience AI MachineLearning DeepLearning Algorithms by Source for graphic: | abdsc (2018.08.02) LINK

“Visible-Near-Infrared Spectroscopy can predict Mass Transport of Dissolved Chemicals through Intact Soil.” (2018.08.02) LINK

“Classification and compositional characterization of different varieties of cocoa beans by near infrared spectroscopy and multivariate statistical analyses.” (2018.08.02) LINK

“Classification of Chicken Parts Using a Portable Near-Infrared (NIR) Spectrophotometer and Machine Learning.” (2018.08.02) LINK

“Rapid Prediction of Low (<1%) trans Fat Content in Edible Oils and Fast Food Lipid Extracts by Infrared Spectroscopy and Partial Least Squares Regression” (2018.07.31) LINK

“Evaluating the performance of a consumer scale SCiO™ molecular sensor to predict quality of horticultural products” (2018.07.30) LINK

“Estimation of Total Phenols, Flavanols and Extractability of Phenolic Compounds in Grape Seeds Using Vibrational Spectroscopy and Chemometric Tools” (2018.07.26) LINK



Near Infrared

“FT-NIR, MicroNIR and LED-MicroNIR for Detection of Adulteration in Palm Oil via PLS and LDA” FTNIR NIRS (2018.08.03) LINK

“Long-Length Fiber Optic Near-Infrared (NIR) Spectroscopy Probes for On-Line Quality Control of Processed Land Animal Proteins” (2018.08.03) LINK

“Near-infrared spectroscopy for rapid and simultaneous determination of five main active components in rhubarb of different geographical origins and processing.” (2018.08.02) LINK

“Marktech Optoelectronics Introduces Silicon Avalanche Photodiodes for Low-Level Light and Short Pulse Detection” UV NIR NIRS SWIR (2018.08.02) LINK

“Innovative Technology Promises Fast, Cost-Efficient Age Data for Fisheries Management” FTNIR (2018.07.31) LINK

“Rapid qualitative and quantitative analysis of methamphetamine, ketamine, heroin, and cocaine by near-infrared spectroscopy.” (2018.07.31) LINK

We (led by ) have been independently assessing thew value of consumer scale NIR devices for horticultural quality assessment. Here is our published work assessing (2018.07.30) LINK



Infrared

“Fault Detection Based on Near-Infrared Spectra for the Oil Desalting Process” (2018.08.05) LINK

“Common Infrared Optical Materials and Coatings: A Guide to Properties, Performance and Applications” (2018.08.04) LINK



Raman

SpectraBase – FREE, fast text access to hundreds of thousands of NMR, IR, Raman, UV-Vis, and mass spectra! spectroscopy (2018.08.02) LINK



Agriculture

“Three-Dimensional Reconstruction of Soybean Canopies Using Multisource Imaging for Phenotyping Analysis” (2018.08.03) LINK

“Smartphone Spectroscopy Promises a Data-Rich Future – An upsurge of portable, consumer-facing devices at the intersection of smartphone computing and spectroscopy is now leveraging integration. ” (2018.08.02) LINK

Innovative Technology Promises Fast, Cost-Efficient Age Data for Fisheries Management (2018.07.31) LINK

“Smartphone-Based Food Diagnostic Technologies: A Review” (2018.07.30) LINK



Petro

“Detection, Purity Analysis, and Quality Assurance of Adulterated Peanut (Arachis Hypogaea) Oils” (2018.08.02) LINK



Pharma

“Potential of near infrared spectroscopy and pattern recognition for rapid discrimination and quantification of Gleditsia sinensis thorn powder with adulterants.” (2018.08.02) LINK



Medicinal

A micro-spectrometer fit for a smartphone: Could the power to measure things like CO2, food freshness, and blood sugar levels soon be in the palm of our hands? |rld/magazine/article/323/micro-spectrometer-opens-door-to-a-wealth-of-new-smartphone-functions?utm_source=twitter.com/CalibModel health safety medicine spectroscopy (2018.02.25) LINK

“Near-infrared spectroscopy detects age-related differences in skeletal muscle oxidative function: promising implications for geroscience.” (2018.02.08) LINK




Other

69% of decision makers say industrial analytics will be crucial for business in 2020. | IoT IIoT MT LINK





CalibrationModel.com

Free Chemometric NIR Predictor Software! Simple plug&play calibrations, drag&drop spectral data, for any NIRS sensor device. Easy to use software for off-line and real-time prediction without limits. offline realtime (2018.08.04) LINK

Automated NIRS spectroscopy chemometrics method development with MachineLearning for spectrometer Spectral IoT sensor SmartSensor SmartSensors (2018.07.25) LINK

Automatic NIR Spectroscopy Calibration-Development as a Service. Applicable with free NIR-Predictor software or via OEM API. | NIRSpectroscopy NearInfrared NIRanalysis spectrometers DataAnalytics Regression Spectral Sensors QualityControl Lab (2018.07.26) LINK

Increase Your Profit with optimized NIRS Accuracy with Calibration as a Service (CaaS) and the new free NIR-Predictor software. | foodsafety Feed Lab QC QA testing (2018.08.03) LINK

New : FREE NIR Predictor Software, drag&drop spectral data to plug&play calibrations, for any NIRS Analysis sensor type. | Chemometrics Prediction (2018.07.24) LINK

Spectroscopy and Chemometrics News (KW 11-30 2018) | NIRS Spectroscopy Chemometrics analysis Spectral Spectrometer Sensors (2018.07.25) LINK

Spektroskopie und Chemometrie Neuigkeiten (KW 11-30 2018) | NIRS Spektroskopie Chemometrie analyse Spektral Spektrometer Sensor (2018.07.25) LINK

Spettroscopia e Chemiometria Weekly (KW 11-30 2018) | NIRS Spettroscopia Chemiometria analisi Spettrale Spettrometro Sensore (2018.07.25) LINK

光谱学和化学计量学新闻(KW#11-#30 2018) | 近红外光谱化学计量学分析光谱仪传感器 (2018.07.26) LINK

分光法とケモメトリックスニュース(KW#11-#30 2018) | 赤外分光法・ケモメトリックスの分光センサーの近く (2018.07.26) LINK




Summary of the NIR Chemometric survey polls

Summary of the NIR Chemometric survey polls (as of end of Sept. 2013)

The interesting finding is that most of the answers fit the following pattern. The most companies that use NIR have one NIR Instrument and only one employee that is able to develop NIR calibrations. For that the most common off-the-shelf chemometrics program is used and spent 2 hours or over a month and therefore gets no calibration training about the complex topics like Chemometrics and NIR Spectroscopy or only once (introduction). The calibration maintenance ranges from never to 3 times a year. Interestingly, there was no one who uses portable NIR instruments. We continue our surveys, for the discovery of new trends. Conclusion Seeing this picture, we think that there is huge potential to improve the calibrations. Advanced knowledge can help individuals to build the calibrations with best practices and improve their models accuracy and reliability. Once the decision and investment in NIR technology is done, you should get the best out of your data, because this extra NIR performance can be given by calibration optimization. We offer this as an easy to use and independent service.

What is a NIR calibration used for?

NIR calibrations are used for NIR contents analysis as a productive analytical method. That is a two step procedure.
  1. A NIR analyzer does a non-destructive optical scan of a sample that yields a measured spectrum in seconds.
  2. A NIR calibration model can quantitatively predict (analyze, determine, estimate) multiple constituents, ingredients, contents, analytes, assay, API and other parameters and attributes (chemical, physical, biological, biochemical, sensory) summarized as properties, out of a single spectrum in milli seconds.
The NIR analysis is a very fast non-destructive analysis method that can replace or backup slower methods like wet chemical analysis, chemistry laboratory, sensory panels or rheology (viscosity). Or a NIR calibration can open the door to new possibilities of analytics, quality assurance and process control, by developing calibration models for parameters that seems to be impossible, because they are based on human knowledge, empirical values or sensory like taste value. If you have an NIR instrument, you can measure your samples systematically and thus develop your own calibration models.

What are pre-developed NIR pre-calibrations?

There are a lot of terms that means the same, pre-calibration or NIR starter calibration or pre-built calibration or pre-installed calibration orcalibration package or pre-developed calibrations or pre-calibrated NIR or global calibrations or nir global calibration package or factory calibrations or universal near-infrared (NIR) calibrations or local calibrations or ready-to-use NIR calibrations or off-the-shelf calibrations or factory-calibrated or pre calculated model or start-up calibrations or calibration equations or prefabricated nir calibrations or calibration library or mathematical model. That are Calibration models that are prepared and developed by a calibration specialist. They have collected a lot of samples over years and measured them with NIR and analyced it with reference methods. The NIR spectra are then calibrated against the reference values. This is called a NIR calibration or calibration model or sometimes calibration curve or calibration equation. Normally a precalibration is delivered as a file that is compatible to the used NIR analysis software. Such a calibration file does not contain the spectra nor the reference values.

So how can that work?

The only thing that is in the file is a description what it is for (e.g. protein in feed) and the chemometric model that is represented and stored as list of vectors and matrices. You can’t visualize them, it’s a black-box file. You have no insight of how the calibration is done, how are the settings, how is the prediction performance. You can not extend the calibration with your data to adjust it to your purpose or specialty. Most often the pre calibration files are protected, so you can use it only with a paid license to your software or even to your instrument serials number. These are some (not well known) limitations you will discover if you got one. But such starter calibrations are very useful to have a fast and easy start with a new NIR spectrometer. That’s the main reason why pre-calibrations are available. The second reason is that a collection of spectra can be reused to build such pre calibrations.

Predicting the future?

Are very old spectra useful to predict the future? To adjust a calibration model with newly collected data, the calibrations grows and contains more and more redundancy. That means there are very similar spectra with the same concentration range. So which spectra can be removed to make the calibration better? You maybe never ask this because often you hear, that the more spectra you put into a model the better it will be. Why to remove some spectra?
  • reduce not needed redundancy
  • makes the calibration smaller and less complex
  • makes the calibration better fit to the current situation of now and the near future
  • remove long past seasonal data if you have natural products because nature is changing
  • and of course bad outliers should be removed

Custom NIR calibrations

Build your own calibrations that perfectly fit to your specific sample matrix of your products and your preferred raw materials from your local suppliers. Nature grows differently depending on the geographical region, by seasons and year by year. As you know that NIR-Spectroscopy is not an absolute method, then you have to think about to calibrate these current changing effects into your models. If you own the spectra and the reference values then your are able to build your own calibration models and re-calibrate them when needed. So you have the full control on Calibration updates (also known as moving models).

Conclusion

A NIR-instrument can only measure NIR spectra. So the usefulness of NIR comes in with calibrations. That is very important to know when buying such an instrument. For a fast start you can use pre-built calibrations. Good reliably calibrations are offered from third party to quite high prices that level is similar to a cheaper NIR-Instrument! To continue successfully it is highly recommended to develop your own customized calibration (multivariate calibration model) with your own data from your own products, especially with the use of natural resources. Therefore you need knowledge about chemometrics and multivariate analysis (MVA), spectroscopy and the software used to get the calibration optimized. It is worthwhile to create your own calibrations, because you can calibrate product characteristics that are not covered by the proposed pre-calibrations.

What is JCAMP-DX ?

JCAMP-DX is a Electronic Data Standards for long-term storage and transfer of chemometric information. The standard is development by the International Union of Pure and Applied Chemistry (IUPAC).

JCAMP-DX is an abreviation for the Joint Committee on Atomic and Molecular Physical data – Data eXchange.

It is an human readable file format that is used to store near infrared spectrometry data (and others like Raman, UV, NMR, mass, x-ray, chromatograms, thermograms) and related chemical and physical information and is used since the late 80s.

Almost all NIR-software packages can export the spectra including the reference values as JCAMP-DX. A single file can contain multiple spectra and reference values. A JCAMP file name looks like “sample.dx“, “sample.jdx” or “sample.jcm“.

All data are stored as labeled fields of variable length using printable ASCII characters. Such files can be loaded in an text editor to check the content:


##TITLE= Indene (FILE: AFIR2.DX)
$$ FILE AFIR2.DX ( derived from TFIR2.DX)
$$ ABSORBANCE
$$ FIXED FORM
$$ INCREASING ABSCISSA
$$ RATIONAL ABSCISSA SPACING
##JCAMP-DX= 4.24 $$ Encoded by INTTODX 1.04 (RS McDonald)
##DATA TYPE= INFRARED SPECTRUM
##ORIGIN= JCAMP-DX Test Disk 1.04
R.S.McDonald, 9 Woodside Dr., Burnt Hills, NY 12027, 518-399-5145
##OWNER= Public Domain
##RESOLUTION= 2.0
##DELTAX=1.00000000
##XUNITS= 1/CM
##YUNITS= ABSORBANCE
##XFACTOR= 1.000000000
##YFACTOR= 0.000100000
##FIRSTX= 400.000
##LASTX= 4.000E+03
##NPOINTS= 3601
##FIRSTY= 3.487E-1
##XYDATA= (X++(Y..Y))
400 3487 3355 3264 3198 3153 3143 3182 3298 3520 3845 4262
411 4783 5449 6304 7383 8684 10209 12041 14123 16003 16162 14191
422 11791 9674 7943 6540 5406 4528 3874 3397 3045 2780 2584
433 2446 2354 2290 2246 2212 2187 2165 2135 2087 2022 1945
444 1865 1786 1713 1649 1596 1550 1512 1478 1448 1422 1401
:

The standard can be downloaded here: “JCAMP-DX: A Standard Form for the Exchange of Infrared Spectra in Computer Readable Form“, ROBERT S. McDONALD and PAUL A. WILKS, JR., Appl. Spectrosc. 42(1), pp151-162, 1988

What is NIR-Spectroscopy? (simple explanation, simply explained)

In the most cases a simple Halogen lamp emits light including the near infrared (NIR) spectrum (harmless radiation) to the sample/probe and the reflected light is measured. The light loses some energy on-and-in the sample depending on its physical and chemical (molecular) structure. The missing part of the light is treated as a fingerprint of the sample that is mathematically analyzed with prefabricated NIR calibration models (built with chemometric methods), based on trained known samples. That makes it possible to simultaneous analyze multiple physical- and chemical-properties (constituent, ingredient, analyte) within a few seconds and is non-destructive to samples.

The Ghost Calibrator

To explain our service in an other way, I use an analogy between a book and a calibration. Building good calibrations is like writing a good book (a bestseller). You can write in a foreign language (chemometrics) with a high sophisticated word-processor (the chemometric software) that has a grammar checker (an outlier detection). Due to the complexity of the language (chemometrics) and the difficulty of the chosen book topic (the data) and the incomplete automatic grammar checker, you can never be sure if the grammar is correct and may not lead to misunderstanding (bad prediction performance). So the best way is to let a native language speaker check and correct the text. In that way (the analogy), you can see us even as a ghostwriter (a ghost calibration developer, a ghostcalibrator) that helps you, writing the book (with long year experience, consolidated knowledge, time saving, a lot of benefit). The analogy fits very well, because you can define the topic of the book (with your data). Finally you own the calibration and you have the full insight in how it is done. You have it under full control.