IoT – Testing Big Data Collection and Storage to the limits

Internet of Things showing images of appliances connected

Posted on Friday, October 16, 2015

The era of smart sensors is upon us, as businesses turn to technology to automate the collection of environmental and operational data. Known as the ‘Internet of Things’ (IoT), these sensors, and the data they accumulate, can be mined to reveal trends and aberrations that inform business strategy, or opportunities for new efficiencies.

IoT spending is expected to top $1.7 trillion by 2020, underscoring the importance of the technology; the data generated by IoT will help to provide additional context for existing Big Data programs, another layer for analysis and action.

Big Data just got bigger

The key challenge behind the success of IoT deployments will be building a platform capable of storing and processing vast amounts of data. Many organisations are already struggling to balance the need for seemingly infinite storage and processing capacity, with limited IT budgets, and the IoT is set to further complicate matters.

Although industry hype suggests that Cloud services are the only solution to this conundrum, concerns about security, privacy and sovereignty cast doubt on the wisdom of such a move. In many cases the best approach would be to build a scalable, Cloud-like platform internally, perhaps using a Cloud service to provide additional processing in the short-term to handle peaks in demand.

Businesses will also need to set aside adequate resources to process and analyse the data collected by sensors. As technology continues to evolve, expect to see the increased use of machine learning and artificial intelligence to create systems capable of acting autonomously based on the available data.

In the beginning…

As initial IoT deployments get under way, businesses can push existing IT assets back into operation for the purpose of providing low cost storage. In most cases, working post-warranty storage hardware is perfectly adequate for the task of housing IoT sensor data, and can often be upgraded to provide maximum capacity at far lower cost than purchasing all new systems.

Once Big Data/IoT needs begin to outstrip the capacity and capability of the older older hardware, it can instead be pushed into service for holding archive data. The Internet of Things may be the future, but post-warranty hardware could play an important part in getting your strategy underway now.