IoT – Testing Big Data Collection and Storage to the limits
Posted on Friday, October 16, 2015
The era of smart sensors is upon us, as businesses turn to technology to automate the collection of environmental and operational data. Known as the ‘Internet of Things’ (IoT), these sensors, and the data they accumulate, can be mined to reveal trends and aberrations that inform business strategy, or opportunities for new efficiencies.
IoT spending is expected to top $1.7 trillion by 2020, underscoring the importance of the technology; the data generated by IoT will help to provide additional context for existing Big Data programs, another layer for analysis and action.
Big Data just got bigger
The key challenge behind the success of IoT deployments will be building a platform capable of storing and processing vast amounts of data. Many organisations are already struggling to balance the need for seemingly infinite storage and processing capacity, with limited IT budgets, and the IoT is set to further complicate matters.
Although industry hype suggests that Cloud services are the only solution to this conundrum, concerns about security, privacy and sovereignty cast doubt on the wisdom of such a move. In many cases the best approach would be to build a scalable, Cloud-like platform internally, perhaps using a Cloud service to provide additional processing in the short-term to handle peaks in demand.
Businesses will also need to set aside adequate resources to process and analyse the data collected by sensors. As technology continues to evolve, expect to see the increased use of machine learning and artificial intelligence to create systems capable of acting autonomously based on the available data.
In the beginning…
As initial IoT deployments get under way, businesses can push existing IT assets back into operation for the purpose of providing low cost storage. In most cases, working post-warranty storage hardware is perfectly adequate for the task of housing IoT sensor data, and can often be upgraded to provide maximum capacity at far lower cost than purchasing all new systems.
Once Big Data/IoT needs begin to outstrip the capacity and capability of the older older hardware, it can instead be pushed into service for holding archive data. The Internet of Things may be the future, but post-warranty hardware could play an important part in getting your strategy underway now.
With AWS Snowball has Amazon just forced your hand in the Big Data Debate?
The most attractive aspect of cloud computing – the ability to access an infinite amount of storage and processing power for big data analysis – has always been something of an insider joke.
EMC joins the race for Private Cloud
EMC has finally released details of its new ScaleIO solution, the fruit of an acquisition completed last year. ScaleIO allows organizations to build a ‘hyper converged infrastructure’ using standard x86 server hardware.