LOBSTER was a European network monitoring system, based on passive monitoring of traffic on the internet. Its functions were to gather traffic information as a basis for improving internet performance, and to detect security incidents.
The project originated from SCAMPI, a European project active in 2004–5, aiming to develop a scalable monitoring platform for the Internet. LOBSTER was funded by the European Commission and ceased in 2007. It fed into "IST 2.3.5 Research Networking testbeds", which aimed to contribute to improving internet infrastructure in Europe.[1]
36 LOBSTER sensors were deployed in nine countries across Europe by several organisations. At any one time the system could monitor traffic across 2.3 million IP addresses. It was claimed that more than 400,000 Internet attacks were detected by LOBSTER.[2]
LOBSTER was based on passive network traffic monitoring. Instead of collecting flow-level traffic summaries or actively probing the network, passive network monitoring records all IP packets (both headers and payloads) that flow through the monitored link. This enables passive monitoring methods to record complete information about the actual traffic of the network, which allows for tackling monitoring problems more accurately compared to methods based on flow-level statistics or active monitoring.
The passive monitoring applications running on the sensors were developed on top of MAPI (Monitoring Application Programming Interface),[3] an expressive programming interface for building network monitoring applications, developed in the context of the SCAMPI and LOBSTER projects. MAPI enables application programmers to express complex monitoring needs, choose only the amount of information they are interested in, and therefore balance the monitoring overhead with the amount of the received information. Furthermore, MAPI gives the ability for building remote and distributed passive network monitoring applications that can receive monitoring data from multiple remote monitoring sensors.
The LOBSTER sensors operated by the various organisations monitored the network traffic using different measurement applications. All applications were developed within the LOBSTER project using MAPI, according to the needs of each organisation.