Wednesday, December 2, 2015

Random projection in a nutshell

I was reading the random projection paper – written by Teija Seitola et al. – the other day, and I think I have now quite a good understanding of how the method works.

The whole thing is based on Johnson–Lindenstrauss lemma from 1984. The lemma says that random projection (nearly) preserves the distances.

Let X be a n×d-dimensional data matrix, where n is the number of data (or ensemble) points and d is the actual dimension. Let f be d×k-dimensional random projection matrix. Let xi and xj be d-dimensional data vectors and ε>0. Now we have that

(1-ε)×ǁxi-xjǁ2 ≤ ǁf(xi)-f(xj2 ≤ (1+ε)×ǁxi-xjǁ2.

In the paper there are also to explicit formulas for k that surprisingly do not depend on the dimension d, but only on the number of observations n and the error ε. For example, k > 4×(ε2/2-ε3/3)-1×log(n).

So if, for example in correlation dimension, we set n = 2000 and ε = 0.1, we need k that is higher than 4*(.1^2/2-.1^3/3)^(-1)*log(2000) = 6515.1. I tested this with 320-dimensional Lorenz and set k to 10000 and the thing works since the curves became so similar (see figure below).



As a conclusion, I would say that random projection method works “better” in large than small dimensional problems, and that the dimension have to be really large before it is worth to use random projections in applications where the preservation of distances is very very important. Note that there are other kinds of applications too…

Below, there is a little Matlab example.

Ciao,
Janne


%% Random projection Matlab example

n = 2000; % Number of time points
d = 20000; % Dimension
k = 10000; % Random dimension

X = 10+5*randn(n,d); % Data matrix

R = randn(d,k); % Random projection matrix
for kkk = 1:d
   R(kkk,:) = R(kkk,:)/norm(R(kkk,:)); % Normalization
end

P = X*R; % Projection

% Distances
ii = 10; jj = 20;
norm(X(ii,:)-X(jj,:))

norm(P(ii,:)-P(jj,:))

Friday, November 27, 2015

Volcanic Sunsets

Hello,

just wanted to share an interesting Youtube video—Alan Robock's 5-minute Ignite talk at NCAR.

Janne

Tuesday, October 27, 2015

OCO-2 target measurements over Sodankylä TCCON station – 20 August 2015

Hello,

I wanted to share an animation showing what  NASA's OCO-2 instrument measured on target mode over Sodankylä TCCON site on 20 August 2015:


Sodankylä is one of the so-called OCO-2 target stations, i.e., once in awhile during one orbit, the instrument measures back and forth over Sodankylä. All those OCO-2 XCOmeasurements shown in the animation were performed in about four minutes. During that day Sodankylä's TCCON FTS instrument measured XCO2 of about 394.5 ppm.

Cheers,
Janne

Sunday, July 26, 2015

2015 Dragon 3 Symposium and best poster award

Hello,

before summer holidays I had the pleasure to participate 2015 Dragon 3 Symposium held in beautiful city of Interlaken, Switzerland. Dragon program is about co-operation between ESA member states and China. The symposium had good atmosphere, and I personally was delighted to receive "best poster award" for my poster presentation. Thanks! Feels good.

As I mentioned the city of Interlaken  between two lakes  is beautiful and surrounded by great mountains. The mayor of Interlaken gave a nice opening to the symposium and told that these days Asians are the most important tourist group in the region, and they come to Switzerland to visit mountains and to buy watches. This fact can clearly be seen in Interlaken, although, I believe that this time around there were even more Chinese people than usual!

On the way back to Zürich Airport I had a chance to visit Bern, and the Einsteinhaus in particular. In that little apartment he wrote the "Annus Mirabilis papers". The one on the photoelectric effect got him Nobel Prize in 1921.

Below are some photos.

Ciao,
Janne

The symposium plenaries were held in a very fancy room. Probably an old theatre.
The workshops were held in normal rooms. Here Erkki is presenting something fancy with Einstein in the middle.
Nice view! 
These skydivers were everywhere!
The Einsteinhaus – right.
The Einsteinhaus  left.
The Einsteinhaus – center. The studio apartment was actually really small.
The Einsteinhaus – window.
Oh, and the "best poster award"!

Wednesday, July 22, 2015

Smoke to Europe from East and West

Hi guys,

SAMPO observed this nice little thing


Those clouds are smoke from Alaska and Canada, and came here from East and West. See also the "Smoke Goes Around the World" by NASA's Earth Observatory website.

Oh, and Happy Summer!

Janne

Monday, June 22, 2015

Sulfur dioxide from Iceland to Finnish Lapland (Bárðarbunga-Holuhraun, September 2014)

Hello,

here are the FMI "science news" of our SO2 paper. First in Finnish:
Satelliitti-havainnot varmistavat: Bárðarbunga-Holuhraun tulivuoren päästöt kulkeutuivat Islannista Suomeen viime syksynä
Satelliitti-instrumentit ja maapinnan mittalaitteet havaitsivat korkeita ilmakehän rikkidioksidipitoisuuksia Lapissa.
Islannin ilmatieteen laitos raportoi 16. elokuuta 2014 kasvavasta vulkaanisesta toiminnasta Bárðarbunga-tulivuoren lähistöllä. 31. elokuuta purkaus alkoi Bárðarbungan koillispuolella sijaitsevasta Holuhraun-halkeamasta. Tästä ns. rakopurkauksesta syöksyi ilmakehään runsaasti rikkidioksidia, mutta ei juuri lainkaan tuhkaa.
Rikkidioksidi (SO2) on pitkäikäinen kaasu, joka voi kulkeutua kauas päästölähteestä, kuten tulivuoresta. Rikkidioksidi on suurina pitoisuuksina myrkyllistä maanpinnan lähellä ja vaikuttaa ilmanlaatuun ja ihmisten terveyteen.
Koska rikkidioksidin määrä on normaalitilanteissa ilmakehässä vähäinen, sitä voidaan käyttää tulivuoren päästöjen seurantaan.
Kohonneita SO2-pitoisuuksia havaittiin satelliitti-instrumenteilla Islannissa 1. syyskuuta 2014 (punaiset alueet kuvassa). Seuraavina päivinä tulivuoresta peräisin olevaa rikkidioksidia kulkeutui itään ja saapui Suomeen 5. Syyskuuta. Tämän jälkeen, korkeita SO2-pitoisuuksia havaittiin Pohjois-Suomessa useita kertoja syyskuun aikana. Lapissa tulivuoriperäiset rikkiyhdisteet aiheuttivat pahaa hajua ja myös näkyvyys oli paikoin heikentynyt. Terveydelle tästä ei aiheutunut haittaa.
Julkaistussa tutkimuksessa käytettiin NASAn EOS-Aura satelliitin OMI- (Ozone Measurement Instrument) sekä Suomi-NPP-satelliitin OMPS (Ozone Mapping and Profiler Suite) -instrumenttien tekemiä rikkidioksidihavaintoja. Operatiiviset OMI ja OMPS SO2-tuotteet kattavat koko maapallon ja ovat saatavilla muutamia tunteja satelliitin ylilennon jälkeen. OMI:n ja OMPS:in rikkidioksidihavaintoja vastaanotetaan myös Sodankylän kansainvälisellä satelliittipalveluasemalla. Näitä Pohjois-Euroopan alueelta saatuja rikkidioksidihavaintoja jaetaan Sampo-palvelun kautta (http://sampo.fmi.fi) noin 15 minuuttia mittausten teon jälkeen.
Tulivuoresta peräisin oleva rikkidioksidi kulkeutui maapinnan lähelle
Kohonneita SO2-pitoisuuksia havaittiin Brewer-instrumentilla Sodankylässä, joka sijaitsee yli 2000 kilometrin päässä Islannin Bárðarbunga-tulivuoresta. Maapinnalta havaitut rikkidioksidiarvot ovat virherajojen puitteissa yhtäpitäviä satelliittihavaintojen kanssa. Satelliittipohjaiset SO2-arvot riippuvat siitä, millä korkeudella SO2-pilvi sijaitsee. Tässä tapauksessa paras yhteensopivuus maapinnalta tehtyjen havaintojen kanssa saatiin, kun satelliittimittauksia prosessoivaan algoritmin asetettiin SO2-pilven korkeudeksi 0,9 kilometriä. Tämä tarkoittaa sitä, että rikkidioksidia oli mahdollisesti myös maanpinnan lähellä. Tämän havainnon vahvistaa se, että kohonneita SO2-pitoisuuksia havaittiin myös ilmanlaadun asemilta Pohjois-Suomessa samoina päivinä kun satelliitti- ja Brewer-mittalaitteet mittasivat korkeita rikkidioksidipitoisuuksia.
Tämä on ensimmäinen kerta, kun maanpinnan lähellä olevasta SO2-pilvestä saatuja satelliittihavaintoja päästiin vertailmaan maanpinnalta tehtyihin havaintoihin. 
Lisätiedot:
Tutkijatohtori Iolanda Ialongo, iolanda.ialongo@fmi.fi

Viite: Ialongo, I., Hakkarainen, J., Kivi, R., Anttila, P., Krotkov, N. A., Yang, K., Li, C., Tukiainen, S., Hassinen, S., and Tamminen, J.: Comparison of operational satellite SO2 products with ground-based observations in northern Finland during the Icelandic Holuhraun fissure eruption, Atmos. Meas. Tech., 8, 2279-2289, doi:10.5194/amt-8-2279-2015, 2015.
www.atmos-meas-tech.net/8/2279/2015/

and then in English:
Satellite observations confirm that Bárðarbunga-Holuhraun volcanic emissions were transported from Iceland to Finland last autumn
Satellite instruments and ground-based devices detected high levels of atmospheric sulfur dioxide concentrations in Lapland.
On August 16, 2014 Icelandic Met Office reported increasing volcanic activity nearby the Bárðarbunga volcano. On August 31 eruption started North-East from Bárðarbunga at the Holuhraun-fissure. This so-called fissure-eruption emitted a large amount of sulfur dioxide into the atmosphere.
Sulphur dioxide (SO2) is a long-lived gas that can be transported far away from the volcanic emission source. Sulphur dioxide is toxic when present in high concentrations near the surface and it can affect air quality and human health. Since the amount of sulfur dioxide in normal situations is very low in the atmosphere, it can be used for monitoring volcanic emissions.
Elevated SO2 concentrations were observed from satellite instruments in Iceland on September 1, 2014 (red areas in the figure). During the following days, the volcanic SO2 was trasported eastward, and reached Finland on September 5. High SO2 concentrations were observed in northern Finland several times during September 2014.
In the study published by Ialongo et al. (2015), SO2 observations from OMI (Ozone Measurement Instrument) onboard NASA's EOS-Aura satellite as well as OMPS (Ozone Mapping and Profiler Suite) on-board Suomi-NPP satellite were used. The operational SO2 products are available globally, and typically within a few hours after the satellite overpass. In addition, OMI and OMPS sulfur dioxide observations are received from the international satellite-service station in Sodankylä. These SO2 retrievals cover Northern Europe and are distributed via Sampo-service (http://sampo.fmi.fi) about 15 minutes after the measurements are performed.
Ground-based measurements and satellite observations confirm that the volcanic SO2 was transported close to the surface.
Elevated SO2 concentrations were observed from Brewer measurements performed in Sodankylä, which is located more than 2000 km from the Bárðarbunga volcano in Iceland. The SO2 observations from Brewer agree within the errorbars with the satellite observations. Satellite-based SO2 values depend on the height of the SO2 cloud. In this case, the best agreement with the ground-based retrievals is obtained when a SO2 cloud height of 0.9 km is assumed in the satellite retrieval algorithm. This means that the volcanic SO2 was probably located close to the surface. This is confirmed by the fact that elevated levels of SO2 concentrations were also observed at several air quality stations in northern Finland during the same days when the satellite and Brewer instruments measured high concentrations of sulfur dioxide. This is the first time when satellite observations of volcanic SO2-plume located very close to the surface were compared with ground-based data, including air quality measurements.
OMI SO2 maps during the Holuhraun eruption in September 2014. The dates (day/month) are indicated in the title of each panel. The blue crosses indicate the location of Sodankylä. Red areas correspond to elevated SO2 levels.
Additional Information:
Postdoc researcher Iolanda Ialongo, iolanda.ialongo@fmi.fi
Reference: Ialongo, I., Hakkarainen, J., Kivi, R., Anttila, P., Krotkov, N.A., Yang, K., Li, C., Tukiainen, S., Hassinen, S., and Tamminen, J.: Comparison of SO2 operational satellite products with ground-based observations in northern Finland during the Icelandic Holuhraun fissure eruption, Atmos. Meas. Tech., 8, 2279-2289, doi: 10.5194 / amt-8-2279-2015, 2015.
www.atmos-meas-tech.net/8/2279/2015/
OMI SO2 maps during the Holuhraun eruption in September 2014. The dates (day/month) are indicated in the title of each panel. The blue crosses indicate the location of Sodankylä. Red areas correspond to elevated SO2 levels.

All the best,
Janne

Friday, June 5, 2015

Island on Fire

Hello,

as I mentioned earlier, the final version of our SO2 paper came out. Great!

Lately, I have been reading about another Icelandic volcanic eruption — Laki A.D. 1783.


Laki was a volcanic eruption that ”turned eighteenth-century Europe dark”. It cause death and famine across Europe and Egypt and was ”so devastating people ate their own shoes”.

Recently, Anja Schmidt has estimated that if a similar eruption would occur today, approximately 142,000 additional cardiopulmonary fatalities (with a 95% confidence interval of 52,000–228,000) could occur in Europe.

The book, Island on Fire, is quite a good read. In addition to Laki, it also discusses about volcanic eruptions in general.


In the book there was one very powerful image from BerkeleyEarth.org


The figure shows quite nicely what is the impact of a ”large” volcanic eruption on ”average Earth land temperature” and how long does it last. Locally, the impact can of course be much greater. And we can even have a ”year without summer”.

Ciao,
Janne

Thursday, June 4, 2015

ATMOS 2015, part II

Hello,

next week there will be ATMOS 2015. I am not participating, but Johanna will present my poster. I also wrote a little proceeding paper. You can read it here.

In addition, our chaos paper ”Generalized correlation integral vectors: A distance concept for chaotic dynamical systems” came out. And final version of the SO2 paper too. I will blog more about these two later.

Good week, huh!?

Janne

Wednesday, February 18, 2015

ATMOS 2015: Validation and Alternative Retrievals of GOMOS Ozone Profiles in the UTLS Altitude Region

Hello,

submitted the abstract below to the The European Space Agency's ATMOS 2015 – Advances in Atmospheric Science and Applications conference. Let's hope it'll get accepted...

Janne
Validation and Alternative Retrievals of GOMOS Ozone Profiles in the UTLS Altitude Region 
Janne Hakkarainen, Iolanda Ialongo, Viktoria Sofieva, Marko Laine, Johanna Tamminen, Erkki Kyrölä 
Finnish Meteorological Institute, Finland 
Global Ozone Monitoring by Occultation of Stars (GOMOS) is a satellite instrument onboard the ENVISAT platform that was in operation during 2002–2012. During this period, GOMOS observed about 880 000 vertical profiles of ozone, NO2, NO3 and aerosols. About half of these measurements were made during nighttime. The GOMOS measurement principle is based on the stellar-occultation technique. 
In this paper, we present rigorous validation and characterization of operational GOMOS ozone profiles in the upper troposphere lower stratosphere (UTLS) altitude region. The GOMOS profiles are validated using ozone soundings from NDACC (Network for the Detection of Atmospheric Composition Change). The results show a strong ozone overestimation by GOMOS in the tropopause region and below. The median relative difference grows up to 100 % and is particularly large in the tropics. The influence of retrieval uncertainties and star properties on the high bias in the troposphere is also investigated. 
In addition to the validation and characterization of the operational data product, we also show recent advantages of alternative retrieval algorithms that are designed in particular for processing the GOMOS measurements for UTLS applications. The retrievals are performed using re-designed two-step algorithm and alternative one-step algorithm. In one-step retrieval, the spectral and the vertical inversions of the two-step algorithm are executed simultaneously. This approach allows a better use of the smoothness prior information and the prior given for example to aerosol parameters affects the other species too. This feature is critical when going near the detection limit. The preliminary results show drastic improvement of the quality of the GOMOS profiles in the UTLS altitude region when compared against NDACC ozone soundings. To further evaluate the novel UTLS dataset (consisting of 171 233 profiles form 49 brightest starts in full dark), we perform a comparison against OSIRIS ozone dataset, which has shown reliable results in the UTLS altitude region.

Saturday, January 31, 2015

Kilpisjärvellä Planetarium film

Hey guys,

This Thursday I went to see Planetarium films in Heureka. The event was part of DocPoin film festival.
Even the sky won’t be the limit when DocPoint extends the festival programme into a new venue: Heureka. The Heureka Planetarium is one of the most modern FullDome theaters in Europe. In FullDome the films are projected onto a hemisphere-shaped silver screen. Experience the documentaries in a whole new way!
There were four short films:
Domefest Demo (3 min)
Losing the Dark (USA, 2013, 12 min, prod. International Dark-Sky Association, Loch Ness Productions)
Starry skies are a becoming a rare sight as light pollution is washing away our view of the cosmos. It not only threatens astronomy, it disrupts wildlife, and affects human health. The yellow glows over cities and towns – seen so clearly from out of space – remind us of the wasted energy and money used for lighting up the sky. 
Dynamic Earth (USA, 2012, 24 min, dir.Thomas Lucas)
Dynamic Earth explores the inner workings of Earth’s climate system. With visualizations based on satellite monitoring data and advanced supercomputer simulations, this cutting-edge procuction follows a trail of rnergy that flows from the Sun into the interlocking systems that shape our climate: the atmosphere, oceans, and the biosphere. 
Kilpisjärvellä (Finland, 2012, 17 min, dir Axel Straschnoy, prod. Mark Lwoff, Misha Jaari / Buf0)
Kilpisjärvellä presents the Northern Lights as they are experienced by someone watching them from the ground. We follow two cameramen who are working to record them on video as they walk through the tundra in order to get away from any artificial lights, and they look for the best spot to capture them. We experience the simple issues of their everyday life, such as getting water from under the ice, chopping wood or preparing a meal. Finally, we also see the lights themselves.
The last one was probably the most interesting among these films and it was also the first Finnish FullDome film. The Argentinian director (who lives in Helsinki) was present and after the screening there was a little Q&A. The film was maybe a little bit too ”artistic” for my taste. It was understandable since Kilpisjärvellä was his first FullDome film and the idea was just simply to go in Lapland for three weeks and make movie. In fact, there were only handful of scenes where the FullDome technique was actually usefull. He told that his next film will be more ambitious. Interestingly, he told that there are no videocameras for making FullDome films (at all!?). Instead he used a normal camera and the film was put together from still shots. Here is a 30 second clip from the film:


Clip from the film KILPISJÄRVELLÄ from Bufo on Vimeo.

One thing that I was constantly thinking during the screening was that how would my satellite data look on that planetarium screen…

Ciao,
Janne

Sunday, January 18, 2015

Validation of satellite SO2 observations in northern Finland during the Icelandic Holuhraun fissure eruption

Hello,

Last Friday our paper "Validation of satellite SO2 observations in northern Finland during the Icelandic Holuhraun fissure eruption" was published as a discussion paper in Atmospheric Measurement Techniques. The paper shows the validation results of the satellite SO2 observations from OMI (Ozone Monitoring Instrument) and OMPS (Ozone Mapping Profiler Suite) during the Icelandic Holuhraun fissure eruption in September 2014. The volcanic plume reached Finland on several days during the month of September.


SO2 total columns as seen from OMI TRL product during the Holuhraun fissure eruption for six days in September 2014. The dates (day/month) are indicated in the title of each panel. The blue crosses indicate the location of Sodankylä ground-based station.

The satellite products assuming a priori profile with SO2 predominantly in the planetary boundary layer give total column values close to the ground-based data, suggesting that the volcanic SO2 plume was located at particularly low altitudes. This is connected to the fact that this was a continuous effusive fissure eruption, without explosive activity, and most of the SO2 was emitted into the troposphere.

The analysis of the SO2 surface concentrations at four air quality stations in northern Finland supports the hypothesis that the volcanic plume coming from Iceland was located very close to the surface. The time evolution of the SO2 concentrations peaks during the same days when large SO2 total column values are measured by the Brewer in Sodankylä and enhanced SO2 signal is visible over northern Finland from the satellite maps. This is an exceptional case because the SO2 volcanic emission directly affect the air quality levels at surface in an otherwise pristine environment like northern Finland. 

You can read the paper from this link.

Citation: Ialongo, I., Hakkarainen, J., Kivi, R., Anttila, P., Krotkov, N. A., Yang, K., Li, C., Tukiainen, S., Hassinen, S., and Tamminen, J.: Validation of satellite SO2 observations in northern Finland during the Icelandic Holuhraun fissure eruption, Atmos. Meas. Tech. Discuss., 8, 599-621, doi:10.5194/amtd-8-599-2015, 2015.