Spectroscopy and Chemometrics News Weekly #7-9, 2017

Near Infrared

Researchers using Near-Infrared Spectroscopy determine the age of cannabis at time of harvest via LINK

Panasonic’s new organic sensor can switch between visible and NIR sensitivity NIRS LINK

“Global Process Spectroscopy Market $1.47bn by 2024 – TechAnnouncer” NIRS Raman LINK



Equipment

“Fabry Perot Detectors Pyroelectric Detectors with Spectrometer Functionality” LINK

“How to choose the right spectrometer?” LINK



Chemometrics

Predicting the ethanol potential of wheat straw using near-infrared spectroscopy and chemometrics LINK



Process Control

CEM for all stages of the process control of dairy products | dairyproducts processcontrol pcontrol LINK

NIR spectroscopy for monitoring of industrial manufacturing processes | pcontrol LINK



Environment

Drones go hyper spectral to expose Ningaloo Reef in new detail gis remotesensing LINK

“Opinion: Soil Testing Needs Sensors To Get More Accurate” LINK



Other

Molecular Spectroscopy Market Globally Expected to Drive Growth through 2025 LINK

“Chemical Gas Sensors on the Rise.” MEMS sensor LINK

Seeing Theory – Basic Probability Compound Probability Distributions Statistical Inference Regression LINK



Spectroscopy and Chemometrics News Weekly #48, 2015

Chemometrics

See how chemometrics is helping save historical audio recordings in this article from | NIRS LINK


Near Infrared

Brewing up a laboratory sample – parameters by near infrared (NIR) instrument | beer LINK


Infrared

For a better understanding of the peaks in infrared (IR) and Raman spectroscopy, check out this primer LINK


Raman

Automated analysis of single cells using Laser Tweezers Raman Spectroscopy LINK

Smart sensor detects single molecule in chemical compounds | Raman spectroscopy SERS LINK


Hyperspectral

Drone multi-spectral imaging for field phenotyping of maize | drones farm365 ag farm LINK

Evaluation of Food Quality and Safety with Hyperspectral Imaging (HSI) LINK


Spectral Imaging

Robotic weeder uses multispectral imaging to identify targets. deepfieldrobotics precisionAG LINK


Optics

Spectroscopy Can Head Off Food Safety Crises (BioPhotonics feature) | LINK


Equipment

Airborne Spectrometers Volcanic Emissions | Drone UAV LINK

Modeling the grating spectrometer in a Czerny-Turner monochromator LINK

Hochauflösendes Spektrometer erkennt gestresste Pflanzen -FLORIS “Fluorescence Imaging Spectrometer” LINK


Process Control

Growing Adoption of Process Spectroscopy Across Industries Driving Global Market, According to BCC Research LINK


Environment

Mapping the Spectral Soil Quality Index (SSQI) Using Airborne Imaging Spectroscopy LINK


Agriculture

Airinov, a huge success in France now available in the UK! Come and see us at croptec15 stand 84 LINK!

FDA issues final food biotechnology labeling guidelines for plant foods; discourages “GMO free” claims | GMO LINK


Quantum

Quantum entanglement achieved at room temperature in semiconductor wafers | quantum LINK

Cool stuff coming from : Language-Integrated Quantum Operations: LIQUi|> LINK


Other

Compac’s Spectrim platform pushes sorting technology forward, postharvest, , |:30810,seccion:news,noticia:77487/ via LINK


CalibrationModel.com

Blogged: Spectroscopy + Chemometrics News Weekly KW47-2015 | Molecular Spectroscopy NIRS Chemometrics Raman NIR LINK

Increase Your Profit with optimized NIR Accuracy Food Feed FoodSafety ag agriculture Lab QAQC FDA LINK

Service for Professional custom development of NIR calibrations | NIRS Near-Infrared-Spectroscopy LINK

Services for Optimization of Chemometric Application Methods of Near-Infrared Spectroscopy | NIRS Quality Control LINK


Chemometrics and Near-Infrared Spectroscopy News Weekly #11, 2015

Get the Chemometrics and Spectroscopy News in real time on Twitter @ CalibModel and follow us.


 

Chemometrics

Low-Content Quantification in Powders Using Raman Spectroscopy:..Chemometric Approach to Sub 0.1% Limits of Detection LINK
Check out this SSPC publication! Low-content quantification in powders using Raman spectroscopy LINK
Feasibility Study for Transforming Spectral and Instrumental Artifacts for Multivariate Calibration Maintenance LINK
ICA-Based Algorithm for Automatic Identification of Raman Spectra Applied to Artistic Pigments and Pigment Mixtures LINK
Optimization of NIR spectroscopy based PLSR models for critical properties of vegetable oils in biodiesel production LINK
Using iSPA-PLS and NIR spectroscopy for determination of total polyphenols and moisture in commercial tea samples LINK

Near Infrared

Broadband near-infrared spectroscopy of organic molecules on compact photonic devices LINK
NIR-Online-Systeme eröffnen neue Dimensionen der Qualitätskontrolle im Mischfutterwerk … LINK
Novel method to correct for wood MOE ultrasonics and NIRS measurements on increment cores in Liquidambar styraciflua LINK
Texas Instruments announces industry’s first fully programmable MEMS chipset f/ NIR analysis LINK
Forearm Deoxyhemoglobin and Deoxymyoglobin (Deoxy[Hb + Mb]) Measured by Near-Infrared Spectroscopy (NIRS) LINK
Determination of Copper & Zinc Pollutants in Ludwigia prostrata Roxb Using Near-Infrared Reflectance Spectroscopy LINK

Raman

Using Raman spectroscopy to characterise dental caries LINK
Global raman spectroscopy industry research report for 2015 just published – WhaTech LINK
Continuous Temperature-Dependent Raman Spectroscopy of Melamine and Structural Analog Detection in Milk Powder LINK

Equipment

New Handheld Analyzer Combines FTIR and Raman Technologies – Thermo Scientific Gemini LINK
First and only handheld integrated Raman and FTIR instrument – Thermo Scientific Gemini LINK
How Rebellion Photonics built a business on gas fumes | Hyperspectral LINK
Making a traditional miniature spectrometer smaller. Spark-VIS from LINK
Bruker’s NEW BRAVO – Handheld Raman Spectrometer LINK
Spectroscopy: Shrinking spectrometers permit novel applications. LINK
Live Demo of Bruker’s New HTS-250 FT-NIR Auto Sampling System LINK
Bruker Launches New BRAVO Handheld Raman Spectrometer for Raw Materials Identification LINK

Agriculture

Cheaper Sensors Will Fuel The Age Of Smart Everything LINK
Rambus Lensless Smart Sensor: A Tiny Image Sensor Designed For IoT – LINK

Pharma

NEW GENERATION OF RAMAN – COST EFFECTIVE RAW MATERIAL ID & PRODUCT ID OF PHARMACEUTICALS LINK

Other

Forbidden quantum leaps possible with high-res spectroscopy LINK
Frozen highly charged ions for highest precision spectroscopy LINK
THE SECRET TO HOW CHAMELEONS CHANGE COLOR: NANOCRYSTALS LINK
Spectroscopy Applications Offer Insight on Measurement Challenges LINK
40 darn good reasons to save seeds this year: LINK
Support research, education, extension. GoodAdviceIn4Words LINK!
Twenty one amazing reasons why forests are important LINK

CalibrationModel.com

9 Reasons why near-infrared Spectroscopy Applications need periodic Calibration Maintenance! LINK
Development of quantitative Multivariate Prediction Models for Near Infrared Spectrometers | NIRS NIR SWIR HSI LINK
Do you use Molecular Spectroscopy with Multivariate Regression Models? TimeSaver | … LINK
Do you use a near-infrared Spectrometer with Chemometric Methods? This will save you time NIR NIRS SWIR NIT LINK
Easy NIR Spectroscopy Calibration LINK
Efficient development of new quantitative prediction equations for multivariate data like NIR spectra | spectrum LINK
Efficient development of new quantitative prediction equations for multivariate data like NIRS spectra | AgTech LINK
Erstellung und Optimierung von chemometrischen Auswertemethoden für NIR Spektrometer | nearIR Labor nahinfrarot NIR LINK
Improve Accuracy of fast Nondestructive NIR Measurements by Optimal Calibration | foodquality agtech Lab biotech LINK
Improve Accuracy of fast Nondestructive NIR Measurements by Optimal Calibration | Food Feed FoodSafety mills Lab LINK
Increase Profit with optimized NIR Accuracy QC QA Food Feed Milk Production laboratory pauto process LINK
Increase Profit with optimized NIR Spectrometer Accuracy measure quality control parameters analyze diagnostics LINK
Increase Your Profit with optimized NIR Spectroscopy Analysis Process Control Monitoring LINK
Increase Your Profit with optimized NIR Accuracy Laboratory QC QA Food Feed Aquaculture petfood grain milk LINK
It is very unlikely for a tweet on “Chemometrics” to be retweeted. LINK
News: Chemiometria e Spettroscopia Weekly News 10, 2015 | NIRS Spettroscopia Chemiometria news LINK
News: Chemometrics and Spectroscopy News Weekly 10, 2015 | … NIRS Spectroscopy Chemometrics news Pittcon15 LINK
News: Chemometrie und Spektroskopie Neuigkeiten Wöchentlich 10, 2015 | NIRS Spektroskopie Chemometrie news LINK
Pro Tip: Die NIR-Kalibrierung ist der zentrale Schlüssel zur genauen NIR Messung LINK
Quantitatⅳe Analytical NIR Method Development Serⅵce 4 Quality Control Laboratory&Analytical Laboratories 4 Food QAQC LINK
Service für Professionelle Entwicklung von NIR-Kalibrierungen | NIRS NearInfrared Spektroskopie NahInfraRot LINK
Services for Professional Development of NIR Calibrations | NIRS Near-Infrared-Spectroscopy QA QC Laboratory LINK
Successful Application Development with NIR spectroscopy for quantitative determination LINK
WHITEPAPER: Knowledgebased Chemometric Software for NIR Calibration Modeling | NIRS LINK
free WHITEPAPER: A Novel knowledgebased Chemometric Software for quantitative NIR Calibration Modeling | NIRS LINK

Procedures for NIR calibration – Creation of NIRS spectroscopy calibration curves

Do you know the effect that you prefer to try out their favorite data pretreatments in combination and often try the same wavelength selections based spectra of the visualized?

You try as six to ten combinations until one of them selects his favorite calibration model, to then continue to optimize. Since then suddenly fall to outliers, because it goes in depth, so is familiar with the data, we know now the spectra of numbers of outliers and is familiar with the extreme values.

Now, the focus is on the major components (principal components, Latent Variables, factors) and makes sure not to over-fit and under-fit not to. The whole takes a few hours and finally one is content with the model found.

So what would happen if you all in the beginning tried variants found outliers removed and re-evaluated and compared? The results would be better than that of the previous model choice? One does not try out? Because it is cumbersome and takes hours again?

We have developed a software which simplifies this so that also the number of model variations can be increased as desired. The variants generation is automated with an intelligent control system, as well as the optimization and comparing the models and finally the final selection of the best calibration model.

Our software includes all the usual known data pretreatment methods (data pre-processing) and can combine them useful. Since many Preteatments are directly dependent on the wavelength selection, such as the normalization the determined within a wavelength range of the scaling factors to normalize the spectra so that pretreatments with the wavelength ranges may be combined. So a variety of settings sensible model comes together that are all calculated and optimized. For the automatic selection of the relevant wavelength ranges, different methods are used, which are based on the spectral intensities. Thus, for example, regions with total absorption is not used, and often interfering water bands removed or retained.

Over all the calculated model variations as a summary outlier analysis can be made. Are there any new outliers (hidden outlier) discovered, all previous models can be automatically recalculated, optimized and compared without these outliers.

From this great number of calculated models with the statistical quality reviews (prediction performance) the optimum calibration can now be selected. For this purpose, not simply sorting by the prediction error (prediction error, SEP RMSEP) or the coefficient of determination (coefficient of determination r2), but by several statistical and test values are used jointly toward the final assessment of optimal calibration.

Thus we have created a platform that allows the highly automated work what a man can never do with a commercial software.

We therefore offer the largest number of matched to your application problem modeling calculations and choose the best calibration for you!

This means that our results are faster, more accurate, robust and objective basis (person independent) and quite easy for you to apply.

You have the full control of the models supplied by us, because we provide a clearly structured and detailed blueprint of the complete calibration, with all settings and parameters, with all necessary statistical characteristics and graphics.

Using this blueprint, you can adjust the quantitative calibration model itself in the software you use, understand and compare. You have everything under control form model creation, model validation and model refinement.

Your privacy is very important to us. The NIR data that you briefly provide us for the custom calibration development will remain of course your property. Your NIR data will be deleted after the job with us.

Interested, then do not hesitate to contact us.

NIRS Calibration Model Equation – Optimal Predictive Model Selection

To give you an insight what we do to find the optimal model, imagine a NIR data set, where a NIR specialist works hard for 4 hours in his chemometric software to try what he can with his chemometric-, NIR spectroscopic- and his product-knowledge to get a good model. During the 4 hours he finds 3 final candidate models for his application. With the RMSEP of 0.49 , 0.51 and 0.6. Now he has to choose one or to test all his three models on new measured NIR spectra.

That is common practice. But is this good practice?

And nobody asks, how long, how hard have you tried, how many trial have you done, if this really the best model that is possible from the data?
And imagine the cost of the data collection including the lab analytics!
And behind this costs, have you really tried hard enough to get the best out of your data? Was the calibration done quick and dirty on a Friday afternoon? Yes, time is limited and manually clicking around and wait in such kind of software is not really fun, so what are the consequences?

Now I come to the most important core point ever, if you own expensive NIR spectrometer system, or even many of them, and your company has collected a lot of NIR spectra and expensive Lab-reference data over years, do you spend just a few hours to develop and build that model, that will define the whole system’s measurement performance for the future? And ask yourself again (and your boss will ask you later), have you really tried hard enough, to get the best out of your data? really?

What else is possible? What does your competition do?

There is no measure (yet) what can be reached with a specific NIR data set.
And this is very interesting, because there are different beliefs if a secondary method like NIR or Raman can be more precise and accurate, as the primary method.

What we do different is, that our highly specialized software is capable of creating large amounts of useful calibrations to investigate this limits – what is possible. It’s done by permutation and combination of spectra-selection, wave-selection, pre-processing sequences and PC selections. If you are common with this, then you know that the possibilities are huge.

For a pre-screening, we create e.g. 42’000 useful calibrations for the mentioned data set. With useful we mean that the model is usable, e.g. R² is higher than 0.8, which shows a good correlation between the spectra and the constituent and it is well fitted (neither over-fitted nor under-fitted) because the PC selection for the calibration-set is estimated by the validation-set and the predictive performance of the test-set is used for model comparisons.

Here the sorted RMSEP values of the Test Set is shown for 42’000 calibrations.
You can immediately see that the manually found performance of 0.49 is just in the starting phase of our optimization. Interesting is the steep fall from 1.0 to 0.5 where manually optimization found it’s solutions. A range where ca. 2500 different useful calibrations exist. The following less steep fall from 0.5 to 0.2 contains a lot more useful models and between 0.2 to 0.08 the obvious high accurate models are around 2500 different ones. So the golden needle is not in the first 2500 models, it must be somewhere in the last 2500 models in the haystack.

Sorted RMSEP plot of 42'000 NIR Calibration Model Candidates

That allows us to pick the best calibration out of 42’000 models, depending on multiple statistical evaluation criteria, that is not just the R² or RPD, SEC, SEP or RMSEP, (or Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), Multivariate AIC  (MAIC) etc.) we do the model selection based on multiple statistical parameters.

Dengrogram plot of similar  NIR Calibration Models

To compare the calibration models by similarity it is best viewed with dendrogram plots like this (zoomed in), where the settings are shown versus the models overall performance similarity. In the settings you can see a lot of different permutations of pre-processings combined with different wave-selections.

NIR Calibration Modeling (Part 2)

( to part 1 )

All the below categories are implemented by using multiple different algorithms and formulas which leads to many different calibrations.

Steps in modeling
  • Data Cleaning – (bad data, missing values, duplicate elimination, spectral quality / intensity / noise, input value typing errors, …)
  • Initial Calibration set up – selection of calibration, validation and test samples
  • Wavelengths selection
  • Data preprocessing, pretreatments
  • Method calculation
  • Choosing the number of Principal Components / Latent Variables
  • Validation of calibration model / Statistics of performance – (accuracy, precision, linearity, repeatability, range, distribution, robustness / stability, sensitivity, simplicity, etc.)
  • Outlier examination and removal


The problem of choosing the optimal number of factors to find the optimum between underfitting and overfitting is solved by having multiple methods and protocols implemented leading to multiple calibrations.

The evaluation and the selection of the best calibration is based on many individual statistical values including the most popular RMSEP, SEP, Bias, SEC, R2 and PCs etc.

Results and Reporting

A detailed calibration report is provided detailing the best available calibration containing all calibration parameter settings and statistics of prediction performance of the calibration set, the validation set and the test set. A visual expression of the calibration is provided with the most importance plots.

Our service works with any quantitative NIR spectra data set in the standard JCAMP-DX format and uses mainly PLS and PCR to be compatible with other chemometric calibration software.

Summary of the NIR Chemometric survey polls

Summary of the NIR Chemometric survey polls (as of end of Sept. 2013)

The interesting finding is that most of the answers fit the following pattern. The most companies that use NIR have one NIR Instrument and only one employee that is able to develop NIR calibrations. For that the most common off-the-shelf chemometrics program is used and spent 2 hours or over a month and therefore gets no calibration training about the complex topics like Chemometrics and NIR Spectroscopy or only once (introduction). The calibration maintenance ranges from never to 3 times a year. Interestingly, there was no one who uses portable NIR instruments. We continue our surveys, for the discovery of new trends. Conclusion Seeing this picture, we think that there is huge potential to improve the calibrations. Advanced knowledge can help individuals to build the calibrations with best practices and improve their models accuracy and reliability. Once the decision and investment in NIR technology is done, you should get the best out of your data, because this extra NIR performance can be given by calibration optimization. We offer this as an easy to use and independent service.

Customized NIR Calibrations

Increase Your Profit with optimized NIR Accuracy


We help you to find the optimal settings for higher NIR accuracy and reliability.

You can build your own custom NIR calibration model with this valuable settings.

We offer a quantitative NIR Calibration development and optimization service.

New: White Paper about the details, what’s behind.


Improve NIR Measurement Accuracy

  • going closer to your product specification limits and maximize profitability
  • optimizing your models yield to process optimization and optimizing productivity
  • compete against other NIR vendors in a feasibility study (NIR salesman)

Easy to use

  • compatible with any NIR vendor
  • no installation, no learning
  • quantitative NIR Calibration Development as a Service

Safety

  • help users avoid common pitfalls of method development
  • before you validate and approve your solution for use in production process:
    • check if a better calibration can be found,
    • compare your calibration with other experts solutions.

Speed

  • no cumbersome trial-and-error modeling steps
  • calculation time is spent on our high performance infrastructure
  • fast results, developed calibrations within days

Fix price

  • fix costs, depends only on data size (not hourly rate for service)
  • huge saving in method development costs
  • easy to plan
More benefits, for whom and where, learn more , contact

NIR Calibration Modeling

The majority of NIR calibrations are generated using a small number of different parameter settings and all too often are restricted to the time a user has available, their spectroscopic and chemometric knowledge and their ability (tedious use of the software) to choose and combine all the possible parameter settings required for good calibrations.

There are many published standards and guidelines (protocols) available for developing NIR calibrations from Standards Consortium such as ASTM, EMEA, ICH, IUPAC, ISO, USP, PASG etc. as well as many good recommendations and guidelines found in various textbooks and papers.

The difficulty with so many ‘Protocols’ for the NIR user is to have them all available and in their thought processes during calibration work and in addition to execute, check and challenge all calibrations generated manually. This is time consuming and sometimes boring repetitive work.

To simplify this for the person generating the NIR Calibrations, we have collected the good practices protocols and integrated them into our service that automates the calibration building and evaluation procedures.

to part 2