Methods and algorithms for processing big data using quantum algorithms

Quantum technologies in big data processing.
Quantum technologies offer potential new opportunities in the field of processing large volumes of data [15]. Here are followed some of the ways in which quantum technologies can be applied in this area:
Quantum Data Processing: Quantum computers offer new capabilities for processing large amounts of data. Algorithms developed for quantum computers can efficiently handle complex tasks, such as searching large databases or optimizing routes.
Quantum machine learning: Quantum technologies can be applied to develop new machine learning methods that can handle large amounts of data. Some research suggests that quantum machine learning algorithms may be more efficient at processing big data than classical methods. The concept of quantum computing originated in the 1980s. in the works of Manin [18], Deutsch [12] and Feynman[15]. The original idea was that it would be more efficient to simulate quantum physical and chemical systems on a computer, which is also quantum in nature. The problem of simulation remains relevant today. Later, also in the 1980s and then in the 1990s, it was realized that due to the phenomenon of “quantum parallelism,” a quantum computer allows solving problems not related to physics or chemistry faster. Shor's algorithm [26] allows solving the factorization problem in polynomial time. This is done by using the property of quantum parallelism and reducing the problem to finding the period of a function.
Quantum simulation: Quantum computers can be used to simulate complex systems such as molecular or physical systems. This can be useful for analyzing and modeling large amounts of data, such as in biology or physics.
Quantum Cryptography: Cryptographic methods based on quantum principles offer new possibilities for ensuring security in the processing of big data. For example, quantum cryptography can be used to develop untraceable encryption algorithms or to provide security during data transmission.
Quantum Sensors and Measurements: Quantum technologies can be applied to develop more accurate and sensitive sensors and measurement methods. This can be useful when working with large amounts of data, such as in medical diagnostics or scientific research.
Application of quantum algorithms in big data.
The application of quantum algorithms in big data processing can offer significant improvements in performance and efficiency compared to classical algorithms. The following are several areas where quantum algorithms can be applied:
Big Data Search: Quantum algorithms such as Grover's algorithm [24]can speed up the search process in disordered databases. This can be of great practical value when processing large amounts of data to determine relevant results. In addition, a significant part is occupied by task optimization, machine learning, quantum clustering and grouping, and quantum simulation.
Optimization problems: Quantum algorithms, such as quantum Markov matrix approximation, can be applied to efficiently solve optimization problems on large data sets. This can be useful in areas such as routing, scheduling or optimization of production processes.
All these possibilities of quantum technologies, including the development and application of quantum algorithms in big data, are still in the early stages of development. Further research and development is required to fully realize and exploit the potential of quantum algorithms in big data processing, which is of interest and the possibility of significant progress in this field.

Fig.5 Bloch diagram

The group of local transformations that leaves the entanglement of a system of n quantum bits unchanged is a tensor product of unitary transformations acting on each of the qubits[27].

. (4)

As a result of the action of a group of local transformations on all possible states of a quantum system, we will obtain the states

(5)

each of which can be considered a representative of a set of states obtained from each other by local unitary operations (equivalent in the local sense). We call the set of such states the variety of locally equivalent states.

Thus, the manifold of locally equivalent states can be defined as the factor space of states of the complete system S based on the states of the subsystems Si. For n qubits we get


(6)

where Hn is the Hilbert space of the quantum register;

H1–Hilbert spaces of each of the qubits.
The dimension of the space of locally equivalent states for n qubits is described by the formula


(7)

The general strategy for constructing a quantum algorithm is based on the choice of a unitary transformation operator U of dimension 2k×2k:

(8)

The Deutsch Jozsa problem is a quantum algorithm proposed by David Deutsch and Richard Jozsa whose simplified meaning is the requirement to find whether a function is a constant. The algorithm is based on the phenomenon of quantum entanglement and the principle of superposition, due to which it demonstrates quantum superiority - significantly more efficient operation in comparison with known classical algorithms. Quantum supremacy is the provision of superpolynomial speedup compared to the best known or possible classical algorithm. The term was popularized by John Preskill[23].

In the quantum formulation, the function is calculated in the form of an appeal to a quantum oracle, producing a unitary transformation Uf that acts on the set | x1x2. . . xn ⟩ | y ⟩ of n qubits x1, x1, . . . x1, which act as arguments to the function, as well as the target qubit y, on which the calculations will be reflected. The result of such a transformation for any eigenstate |x⟩ is the set | x⟩ | y ⨂ f (x) ⟩, (where ⨂ is the designation of the exclusive “or”) corresponding to the result of the controlled negation operation of the qubit | y ⟩ using the qubit |f(x)⟩ . Since unitary transformations are linear, the result of this transformation for a superposition of eigenstates is | х⟩ = α₀| 0 ⟩+ . . .+ α₂ⁿ_-1|2ⁿ-1⟩ defined as U_f (| х⟩ | у ⟩)= α₀| 0 ⟩+ . . .+ α₂ⁿ_-1|2ⁿ-1⟩| у ⨂ f (2ⁿ-1) ⟩.
The Deutsch-Jozsa algorithm only needs a single call to the quantum oracle to reliably solve the problem.
This is the state of our data-driven world today. Tremendous advances in the density of computing power and data storage and availability enable entirely new applications and locations for digital technology and services. The resulting demand in turn drives further advances in our ability to collect, manage, process, and deliver data — in context, in step with business workflows, and in the stream of life. The consequence of this recursive cycle is explosive growth in the global datasphere (see Figure 6).


Fig.6 The volume of data generated in the world in zettabytes.


As a result, due to the limited capabilities of classical computing power, as well as processing algorithms that were used before the amount of generated data was within their real-time capabilities. The ability to process them on the threshold of an explosive increase in data flow may no longer meet the needs of the assigned tasks. Therefore, speaking about new approaches in order to even slightly speed up the data processing process, it is reasonable to use quantum algorithms since quantum computers and quantum approaches are the most effective solutions for processing and analyzing big data in real time due to their high speed. In addition, it is known that the most effective approach to processing big data is mechanisms for processing and analyzing data before storage. In this case, data preprocessing through quantum computing provides high efficiency. Owing to the high speed of quantum computing, this process takes very little time, resulting in data stored with and without preprocessing requiring almost the same amount of time. This reflects the effectiveness of quantum approaches.


1 step – Start;
2 step – accepting big data;
3 step – big data preprocessing using quantum approaches;
4 step – getting results;
5 step – storage data;
6 step – end.

Conclusion.

This study examined several sources related to the field of big data. The study analyzed big data and approaches to its storage and processing. Along with this, MapReduce methods for processing big data, machine learning algorithms, NoSQL databases, and the operating principle of YARN were discussed. The general concept of processing and analyzing big data using quantum technologies was also reviewed and it was determined that this approach could be the most effective solution for processing big data in real time.
Refereces:

1. 
   Akhatov A.R. “Methods for controlling the authenticity of textual information transfer on the basis of statistical and structural redundancy” International Journal of Automation and Computing, 2012 9(5), pp. 518–529, doi: 10.1007/s11633-012-0675-8
   
2. 
   Akhatov A.R., Nazarov F.M., Rashidov A. “Increasing Data Reliability by Using Bigdata Parallelization Mechanisms” International Conference on Information Science and Communications Technologies: Applications, Trends and Opportunities, ICISCT 2021a, 2021 doi: 10.1109/ICISCT52966.2021.9670387
   
3. 
   Akhatov A.R., Nazarov F.M., Rashidov A. “Mechanisms of Information Reliability in BigData and Blockchain Technologies” International Conference on Information Science and Communications Technologies: Applications, Trends and Opportunities, ICISCT 2021b, doi: 10.1109/ICISCT52966.2021.9670052
   
4. 
   Akhatov A.R., Sabharwal M., Nazarov F.M., Rashidov A.R. “Application of cryptographic methods to blockchain technology to increase data reliability” 2nd International Conference on Advance Computing and Innovative Technologies in Engineering, 2022 ICACITE 2022, pp. 642–647. Doi: 10.1109/ICACITE53722.2022.9823674
   
5. 
   Akhatov A.R., Nazarov F.M. “The Same Size Distribution of Data Based on Unsupervised Clustering Algorithms” ICAILE 2023: Advances in Artificial Systems for Logistics Engineering III. (LNDECT, volume 180), 437–447 p, doi: https://doi.org/10.1007/978-3-031-36115-9_40
   
6. 
   Axatov A.R., Rashidov A.E. “Big Data va uning turli sohalardagi tadbiqi”, “Muhammad Al-Xorazmiy avlodlari” Ilmiy-amaliy va axborot-tahliliy jurnali, № 4 (18), 12.2021, 135-144.
   
7. 
   Axatov A.R., Rashidov A.E., Nazarov F.M. “Real-Time Big Data Processing Based on a Distributed Computing Mechanism in a Single Server” Stochastic Processes and Their Applications in Artificial Intelligence. IGI Global, May, 2023, DOI:10.4018/978-1-6684-7679-6
   
8. 
   Axatov A.R., Rashidov A.E., Nazarov F.M., Renavikar A. “Optimization of the number of databases in the Big Data processing” Проблемы информатики, № 1(58), 2023, DOI: 10.24412/2073-0667-2023-1-33-47
   
9. 
   Chak Lam Hadoop v deystvii. –M.:DMK Press, 2012. – 424s.:il.
   
10. 
    Chris Bernhardt Quantum Computing for Everyone Cambridge, MA: The MIT Press, 2019
    
11. 
    Clifford Linch 2008. Big data. Volume 455 Issue 7209, 4 September 2008.
    
12. 
    Deutsch D. Quantum theory, the Church-Turing principle and the universal quantum computer, Proc. Roy. Soc. Lond., A 400, 1985
    
13. 
    Dodin D.V. K voprosu ob analiticheskom opisanii lokalno ekvivalentnih kvantovih sostoyaniy ISSN 2222-8896. Vestn.Volgogr. gos.un-ta.Ser. 1, Mat.Fiz. 2011.
    
14.