Spectroscopy and Chemometrics News Weekly #48+49, 2016

Get the Chemometrics and Spectroscopy News in real time on Twitter @ CalibModel and follow us.



Near Infrared

Cannabis Analysis – On-Site Determination of Cannabis Strength using FT-IR Spectrosocopy FTNIR Ingredients LINK

Near Infrared NIRS, GC and HPLC Applications in Cannabis Testing THC CBD LINK

Raman

What happens when you use Raman spectroscopy to discriminate between brands of extra-virgin olive oil LINK

Raman spectroscopy of chocolate bloom LINK


Hyperspectral

Hyperspectral photoluminescence imaging of defects in solar cells | solar cells via LINK


Agriculture

Soy meal Protein bands LINK

Vitamin C distribution in acerola fruit by near infrared hyperspectral imaging HSI LINK


Equipment

Spectroscopists need freedom to analyse their spectral data, uncoupled from spectrometer hardware! LINK!


Chemometrics

Quality parameters in Castanhola fruit by NIRS to development of prediction models using PLS … in laboratory scale LINK

Monitoring Process-Water Quality Using NIRS and PLSR with Prediction Uncertainty Estimation LINK


Food & Feed

NIR diffuse reflection analysis of fruit – Food Science & Technology LINK


Agriculture

Innovation für die Obstwirtschaft: Neue Ansätze zur Messung und Vorhersage der Apfelqualität MONALISA LINK


Other

Hackers beware! Faking 3D-printed products just got harder. Full-spectrum spectroscopy for the win! LINK!

3D NDVI, using a low cost multi spectral camera. LINK

On the Generation of Random Multivariate Data | Multivariate Data LINK


CalibrationModel.com

Spectroscopy and Chemometrics News Weekly 46+47, 2016 | Spectroscopy NIRS Multivariate DataAnalysis Software LINK

Spektroskopie und Chemometrie Neuigkeiten Wöchentlich 46+47, 2016 | NIRS Spektroskopie Multivariate DatenAnalyse LINK

Spettroscopia e Chemiometria Weekly News 46+47, 2016 | NIRS Spettroscopia Chemiometria LINK


Procedures for NIR calibration – Creation of NIRS spectroscopy calibration curves

Do you know the effect that you prefer to try out their favorite data pretreatments in combination and often try the same wavelength selections based spectra of the visualized?

You try as six to ten combinations until one of them selects his favorite calibration model, to then continue to optimize. Since then suddenly fall to outliers, because it goes in depth, so is familiar with the data, we know now the spectra of numbers of outliers and is familiar with the extreme values.

Now, the focus is on the major components (principal components, Latent Variables, factors) and makes sure not to over-fit and under-fit not to. The whole takes a few hours and finally one is content with the model found.

So what would happen if you all in the beginning tried variants found outliers removed and re-evaluated and compared? The results would be better than that of the previous model choice? One does not try out? Because it is cumbersome and takes hours again?

We have developed a software which simplifies this so that also the number of model variations can be increased as desired. The variants generation is automated with an intelligent control system, as well as the optimization and comparing the models and finally the final selection of the best calibration model.

Our software includes all the usual known data pretreatment methods (data pre-processing) and can combine them useful. Since many Preteatments are directly dependent on the wavelength selection, such as the normalization the determined within a wavelength range of the scaling factors to normalize the spectra so that pretreatments with the wavelength ranges may be combined. So a variety of settings sensible model comes together that are all calculated and optimized. For the automatic selection of the relevant wavelength ranges, different methods are used, which are based on the spectral intensities. Thus, for example, regions with total absorption is not used, and often interfering water bands removed or retained.

Over all the calculated model variations as a summary outlier analysis can be made. Are there any new outliers (hidden outlier) discovered, all previous models can be automatically recalculated, optimized and compared without these outliers.

From this great number of calculated models with the statistical quality reviews (prediction performance) the optimum calibration can now be selected. For this purpose, not simply sorting by the prediction error (prediction error, SEP RMSEP) or the coefficient of determination (coefficient of determination r2), but by several statistical and test values are used jointly toward the final assessment of optimal calibration.

Thus we have created a platform that allows the highly automated work what a man can never do with a commercial software.

We therefore offer the largest number of matched to your application problem modeling calculations and choose the best calibration for you!

This means that our results are faster, more accurate, robust and objective basis (person independent) and quite easy for you to apply.

You have the full control of the models supplied by us, because we provide a clearly structured and detailed blueprint of the complete calibration, with all settings and parameters, with all necessary statistical characteristics and graphics.

Using this blueprint, you can adjust the quantitative calibration model itself in the software you use, understand and compare. You have everything under control form model creation, model validation and model refinement.

Your privacy is very important to us. The NIR data that you briefly provide us for the custom calibration development will remain of course your property. Your NIR data will be deleted after the job with us.

Interested, then do not hesitate to contact us.

NIR Spectroscopy Calibration Report for quantitative predictive models

When you send your quantitative NIR spectra data to our NIR Calibration Model Service, you get a detailed calibration report (calibration protocol) of the found optimal calibration settings, so you are able to see all insights and easily re-build the model in your NIR/Chemometric software.

Here is a part of our calibration report, that exactly describes the data used in the calibration set (CSet), the validation set (VSet) and the test set (TSet). The numbers are the number ids of the spectra in your delivered NIR data file.


The calibration method settings and parameters are
Waveselection : the variable selection or wavenumber selection or wavelength selection
Pretreatments : the spectral data pre-processing
PCs : the number of  Principal Components (PC) or Latent Variables (LV)
Method : the modeling method algorithm used, e.g. PLS

Then the statistical analysis of the PLS model by the different sets (CSet, VSet, Tset).

Calibration Report

Statistical analysis of calibration, validation and test results : 1 Name, 2 Unit, 3 N : number of spectra, 4 N : number of samples, 5 Average spectra count per sample, 6 Reference values, 7 Min, 8 Mean, 9 Median, 10 Max, 11 Standard deviation, 12 Skewness : left (-) or right (+) lack of symmetry, 13 Kurtosis : flat (-) or peaked (+) shape, 14 Model statistics, 15 RPD, 16 R², 17 RMSEC, RMSEP, RMSET : root mean square of prediction errors, 18 SEC, SEP, SET : standard error (bias corrected), 19 Bias, 20 Skewness of prediction errors, 21 Kurtosis of prediction errors, 22 Intercept, 23 Slope, 24 Intercept (reverse), 25 Slope (reverse), 26 Sample Prediction Repeatability Error, 27 Sample Prediction Repeatability Error (of Missing data MSet)

This shows how we deliver the optimal settings. With the statistical values, the NIR model predicted values of all spectra and additional plots you are able to compare with your re-built model to verify that the models perform nearly equally.

Customized NIR Calibrations

Increase Your Profit with optimized NIR Accuracy


We help you to find the optimal settings for higher NIR accuracy and reliability.

You can build your own custom NIR calibration model with this valuable settings.

We offer a quantitative NIR Calibration development and optimization service.

New: White Paper about the details, what’s behind.


Improve NIR Measurement Accuracy

  • going closer to your product specification limits and maximize profitability
  • optimizing your models yield to process optimization and optimizing productivity
  • compete against other NIR vendors in a feasibility study (NIR salesman)

Easy to use

  • compatible with any NIR vendor
  • no installation, no learning
  • quantitative NIR Calibration Development as a Service

Safety

  • help users avoid common pitfalls of method development
  • before you validate and approve your solution for use in production process:
    • check if a better calibration can be found,
    • compare your calibration with other experts solutions.

Speed

  • no cumbersome trial-and-error modeling steps
  • calculation time is spent on our high performance infrastructure
  • fast results, developed calibrations within days

Fix price

  • fix costs, depends only on data size (not hourly rate for service)
  • huge saving in method development costs
  • easy to plan
More benefits, for whom and where, learn more , contact

NIR Spectroscopy and Chemometric surveys, inquiry, polls and assessments (Part 3)


10. NIR in Supply Chain
Where in the supply chain are your NIR instruments located?

11. NIR Usage
How long has your company used NIR spectroscopy?

12. NIR instruments
How many NIR instrument units are in use in your company?

13. NIR Mobile
How much is the mobile hand-held percentage of total NIR devices in your company?

14. Calibration Source
How do you get the NIR Calibrations developed?

15. Calibration Training
How often do the operators get training about NIR Spectroscopy and Chemometrics?

16. NIR PreCalibrations
How many NIR Pre-Calibration, NIR factory calibrations or NIR starter calibrations have you in use?

17. Calibration Spectra
How many Spectra does your quantitative Calibration have in average?

Please vote and see the assessments below.

Part 1, Part 2
NIR in Supply Chain
Where in the supply chain are your NIR instruments located?
NIR Usage
How long has your company used NIR spectroscopy?
NIR instruments
How many NIR instrument units are in use in your company?
NIR Mobile
How much is the mobile hand-held percentage of total NIR devices in your company?
Calibration Source
How do you get the NIR Calibrations developed?
Calibration Training
How often do the operators get training about NIR Spectroscopy and Chemometrics?
NIR PreCalibrations
How many NIR Pre-Calibration, NIR factory calibrations or NIR starter calibrations have you in use?
Calibration Spectra
How many Spectra does your quantitative Calibration have in average?

Part 1, Part 2

NIR Calibration Modeling

The majority of NIR calibrations are generated using a small number of different parameter settings and all too often are restricted to the time a user has available, their spectroscopic and chemometric knowledge and their ability (tedious use of the software) to choose and combine all the possible parameter settings required for good calibrations.

There are many published standards and guidelines (protocols) available for developing NIR calibrations from Standards Consortium such as ASTM, EMEA, ICH, IUPAC, ISO, USP, PASG etc. as well as many good recommendations and guidelines found in various textbooks and papers.

The difficulty with so many ‘Protocols’ for the NIR user is to have them all available and in their thought processes during calibration work and in addition to execute, check and challenge all calibrations generated manually. This is time consuming and sometimes boring repetitive work.

To simplify this for the person generating the NIR Calibrations, we have collected the good practices protocols and integrated them into our service that automates the calibration building and evaluation procedures.

to part 2

NIR Spectroscopy and Chemometric surveys, polls and assessments (Part 2)


5. Calibration Precision
What do you believe, can NIR calibration models be more precise than reference values?

6. Calibration Maintenance
How often do you update your quantitative calibrations per year?

7. Quantitative Calibrations
How many quantitative (%) calibrations do you have in use?

8. Quantitative Parameters
In all your quantitative calibrations, how many parameters (properties) you have in total?

9. Qualitative Calibrations
How many qualitative (identification) calibrations do you have in use?

Please vote and see the assessments below.
Part 1, Part 3
Calibration Precision
What do you believe, can NIR calibration models be more accurate than reference method?
Calibration Maintenance
How often do you update your quantitative calibrations per year?
Quantitative Calibrations
How many quantitative (%) calibrations do you have in use?
Quantitative Parameters
In all your quantitative calibrations, how many parameters (properties) you have in total?
Qualitative Calibrations
How many qualitative (identification) calibrations do you have in use?

Part 1, Part 3

What is a NIR calibration used for?

NIR calibrations are used for NIR contents analysis as a productive analytical method. That is a two step procedure.
  1. A NIR analyzer does a non-destructive optical scan of a sample that yields a measured spectrum in seconds.
  2. A NIR calibration model can quantitatively predict (analyze, determine, estimate) multiple constituents, ingredients, contents, analytes, assay, API and other parameters and attributes (chemical, physical, biological, biochemical, sensory) summarized as properties, out of a single spectrum in milli seconds.
The NIR analysis is a very fast non-destructive analysis method that can replace or backup slower methods like wet chemical analysis, chemistry laboratory, sensory panels or rheology (viscosity). Or a NIR calibration can open the door to new possibilities of analytics, quality assurance and process control, by developing calibration models for parameters that seems to be impossible, because they are based on human knowledge, empirical values or sensory like taste value. If you have an NIR instrument, you can measure your samples systematically and thus develop your own calibration models.

What are pre-developed NIR pre-calibrations?

There are a lot of terms that means the same, pre-calibration or NIR starter calibration or pre-built calibration or pre-installed calibration orcalibration package or pre-developed calibrations or pre-calibrated NIR or global calibrations or nir global calibration package or factory calibrations or universal near-infrared (NIR) calibrations or local calibrations or ready-to-use NIR calibrations or off-the-shelf calibrations or factory-calibrated or pre calculated model or start-up calibrations or calibration equations or prefabricated nir calibrations or calibration library or mathematical model. That are Calibration models that are prepared and developed by a calibration specialist. They have collected a lot of samples over years and measured them with NIR and analyced it with reference methods. The NIR spectra are then calibrated against the reference values. This is called a NIR calibration or calibration model or sometimes calibration curve or calibration equation. Normally a precalibration is delivered as a file that is compatible to the used NIR analysis software. Such a calibration file does not contain the spectra nor the reference values.

So how can that work?

The only thing that is in the file is a description what it is for (e.g. protein in feed) and the chemometric model that is represented and stored as list of vectors and matrices. You can’t visualize them, it’s a black-box file. You have no insight of how the calibration is done, how are the settings, how is the prediction performance. You can not extend the calibration with your data to adjust it to your purpose or specialty. Most often the pre calibration files are protected, so you can use it only with a paid license to your software or even to your instrument serials number. These are some (not well known) limitations you will discover if you got one. But such starter calibrations are very useful to have a fast and easy start with a new NIR spectrometer. That’s the main reason why pre-calibrations are available. The second reason is that a collection of spectra can be reused to build such pre calibrations.

Predicting the future?

Are very old spectra useful to predict the future? To adjust a calibration model with newly collected data, the calibrations grows and contains more and more redundancy. That means there are very similar spectra with the same concentration range. So which spectra can be removed to make the calibration better? You maybe never ask this because often you hear, that the more spectra you put into a model the better it will be. Why to remove some spectra?
  • reduce not needed redundancy
  • makes the calibration smaller and less complex
  • makes the calibration better fit to the current situation of now and the near future
  • remove long past seasonal data if you have natural products because nature is changing
  • and of course bad outliers should be removed

Custom NIR calibrations

Build your own calibrations that perfectly fit to your specific sample matrix of your products and your preferred raw materials from your local suppliers. Nature grows differently depending on the geographical region, by seasons and year by year. As you know that NIR-Spectroscopy is not an absolute method, then you have to think about to calibrate these current changing effects into your models. If you own the spectra and the reference values then your are able to build your own calibration models and re-calibrate them when needed. So you have the full control on Calibration updates (also known as moving models).

Conclusion

A NIR-instrument can only measure NIR spectra. So the usefulness of NIR comes in with calibrations. That is very important to know when buying such an instrument. For a fast start you can use pre-built calibrations. Good reliably calibrations are offered from third party to quite high prices that level is similar to a cheaper NIR-Instrument! To continue successfully it is highly recommended to develop your own customized calibration (multivariate calibration model) with your own data from your own products, especially with the use of natural resources. Therefore you need knowledge about chemometrics and multivariate analysis (MVA), spectroscopy and the software used to get the calibration optimized. It is worthwhile to create your own calibrations, because you can calibrate product characteristics that are not covered by the proposed pre-calibrations.

What is NIR-Spectroscopy? (simple explanation, simply explained)

In the most cases a simple Halogen lamp emits light including the near infrared (NIR) spectrum (harmless radiation) to the sample/probe and the reflected light is measured. The light loses some energy on-and-in the sample depending on its physical and chemical (molecular) structure. The missing part of the light is treated as a fingerprint of the sample that is mathematically analyzed with prefabricated NIR calibration models (built with chemometric methods), based on trained known samples. That makes it possible to simultaneous analyze multiple physical- and chemical-properties (constituent, ingredient, analyte) within a few seconds and is non-destructive to samples.