BDI: An R&D Perspective

By definition, Big Data, is data whose scale, diversity, and complexity require new architecture, techniques, algorithms, and analytics to manage it and extract value and hidden knowledge from it.  In other words, big data is characterised by volume, variety (structured and unstructured data) velocity (high rate of changing) and veracity (uncertainty and incompleteness).

In the Big Data research context, so called analytics over Big Data is playing a leading role. Analytics cover a wide family of problems mainly arising in the context of Database, Data Warehousing and Data Mining research. Analytics research is intended to develop complex procedures running over large-scale, enormous in-size data repositories with the objective of extracting useful knowledge hidden in such repositories. One of the most significant application scenarios where Big Data arise is, without doubt, scientific computing. Here, scientists and researchers produce huge amounts of data per-day via experiments (e.g., disciplines like high-energy physics, astronomy, biology, bio-medicine, and so forth). But extracting useful knowledge for decision making purposes from these massive, large-scale data repositories is almost impossible for actual DBMS-inspired analysis tools. From a methodological point of view, there are also research challenges. A new methodology is required for transforming Big Data stored in heterogeneous and different-in-nature data sources (e.g., legacy systems, Web, scientific data repositories, sensor and stream databases, social networks) into a structured, hence well-interpretable format for target data analytics. As a consequence, data-driven approaches, in biology, medicine, public policy, social sciences, and humanities, can replace the traditional hypothesis-driven research in science.

Big Data: Science & Technology - Challenges

Some of the S&T challenges that researchers across the globe and as well as in India facing are related to data deluge pertaining to Astrophysics, Materials Science, Earth & atmospheric observations, Energy, Fundamental Science, Computational Biology, Bioinformatics & Medicine, Engineering & Technology, GIS and Remote Sensing, Cognitive science and Statistical data. These challenges requires development of advanced algorithms, visualization techniques, data streaming methodologies and analytics. The overall constraints that community facing are

1. The IT Challenge: Storage and computational power
2. The computer science :Algorithm design, visualization, scalability (Machine Learning, network & Graph analysis, streaming of data and text mining), distributed data, architectures, data dimension reduction and implementation
3. The mathematical science: Statistics, Optimisation, uncertainty quantification, model development (statistical, Ab Initio, simulation) analysis and systems theory
4. The multi-disciplinary approach: Contextual problem solving

Big data analytics and the india equation

To tap the analytics momentum, India now needs to build a sustainable analytics eco-system that brings in a strong partnership across the industry players, government, and academia. Some of the key actions for analytics eco-system in India would be around.

1. Talent Pool - Create industry academia partnership to groom the talent pool in universities as well as develop strong internal training curriculum to advance analytical depth.
2. Collaborate - Form analytics forum across organization boundaries to discuss the pain-points of the practitioner community and share best practices to scale analytics organizations.
3. Capability Development - Invest in long term skills and capabilities that forms the basis for differentiation and value creation. There needs to be an innovation culture that will facilitate IP creation and asset development.
4. Value Creation - Building rigor to measure the impact of analytics deployment is very critical to earn legitimacy within the organization.

Big Data and analytics offers tremendous untapped potential to drive big business outcomes. For organizations to leverage India as a global analytics hub can be one of the key levers to move up their analytics maturity curve.

Broad contours of DST initiated BDI programme

• To promote and foster Big Data Science, Technology and Applications in the country and to develop core generic technologies, tools and algorithms for wider applications in Govt.
• To understand the present status of the industry in terms of market size, different players providing services across sectors/ functions, opportunities, SWOT of industry, policy framework (if any), present skill levels available etc.
• To carryout market landscape survey to assess the future opportunities and demand for skill levels in next 10 years
• To carryout gap analysis in terms of skills levels and policy framework.
• To evolve a strategic Road Map and micro level action plan clearly defining of roles of various stakeholders – Govt., Industry, Academia, Industry Associations and others with clear timelines and outcome for the next 10 years.

Call for Proposal under Big Data Initiative Programme (to be opened shortly)

Performa for Submission of Research Projects for Financial Support