Parallel-for loop using OpenMP

6 min readDec 7, 2022

Parallel Computing

First , Let’s know about what is meant by parallel computing and where as parallel computing is not a new thing but it have huge number of applications. There are some recent applications they are taking huge advantage on parallel computing like neural networks ,Bitcoin ,Internet of things(IOT) and so on.. Our article is able to give some interesting matter in the parallel computing for “FOR LOOPS” using OpenMp library.

Formal definition of parallel computing is Splitting of some very big problem statements into smaller or tiny chunks and These parts have to independent of each other. These smaller parts able to execute concurrently by so many multi processors that communicate by the sharing memory in the world of parallel computing. Where as parallel computing exactly looks like this by seeing the below picture.

OpenMP Library

Full form of OpenMP is Open Multi-Processing and it is a process of parallelization of certain amount code section. We will use this Open MP library as the extension for the C ,C++ and Fortran language for addition paralyzed attributes to them. By using this Open MP library we can decrease the time of parallel execution computing and it will take little bit of more time with out this OpenMP library.

We use this library for parallel program execution for the symmetric or shared memory processors and we can tell that this library is typically used loop-level parallelism and We can also Use this for function-level parallelism. The applications of this library are used optimizing of parallelloops,synchronizations,scheduling and in reduction process. Some OpenMP environment's variables :OMP_SCHEDULE,OMP_NUM_SCHEDULE and OMP_DYNAMIC.

Parallel Matrix Multiplication using For loops

We have done so many things on the matrix multiplication from our school days and we are doing this concept still present time also in our studies. As,we already know that if the dimensions of the matrix increases it will take more time for whole calculations of matrix. Even our machines will face problems for doing this big calculations.

Here, In this article we will discuss the “Parallel-For Loop” technique which is used for optimization of matrix multiplications For Performing this technique i will take some random dimensions such as 200 to 2000 with a step of 200

For Performing this Matrix multiplication by using parallel for loops we should need these two requirements and they are

C Language Program
OpenMP Library

Fundamental Matrix Multiplication

Consider the random matrices A and B. Because of matrix are squared, n = m = p.

Result of the above image can be written as shown below:

Just create some random square matrix

Get Vamsidharg’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

We can started by making ignorant matrix for multiplying. In this instance, we’re calling the malloc function to initialize programmatically at the heap memory. Since when it involves testing, we should deal with grids of different diameters.

The data type is defined like double above, but it can be changed based on usage of scenario. where as we used “#Pragma omp parallel” to the line that able to execute different loops of parallelization

Parallel matrix multiplication using For loop

Here,we are trying to implement or execute the the parallelization in our technique that we will use in our multiplication and we also use the library called as OpenMP library as I told about this earlier. Where as this algorithm will make our big work so easy and we can also use our execution with the help of the threads and also it is able handle some features like load balancing race conditions and so on.. But in my article we are able to hold with OpenMP Library for parallelism of matrix multiplication and I have written small snippet of code for parallel matrix multiplication.

Optimized Parallel matrix multiplication using For loop

We have some fundamental issues for the previous execution? As I have not yet completed the storage efficiency in the case. By doing any minimizing access huge brain of memory, the CPU is doing nothing to instruction set on data in a timely manner. Because multiplication of matrices are stored in last in first out memory and also it is not easy task to take those back in the LIFO. It is preferable to move the details to take off from last in first out to first in first out beginning computational procedure. As a result, we must first create containers for that information.

TYPE flatA[MAX_DIM];
TYPE flatB[MAX_DIM];

The steps for implementing efficient mathematical operations are shown below.

Similar Computations at that point

So, many times we just ignore little arithmetic computations that are useless throughout the code and are therefore not required for efficiency but are required for simplicity.

For each repetition of k, the iOff and jOff calculations are identical. So we did the calculations straight away after getting accessibility where we required it.

2.Execution of a cache-friendly method

We all know that recollection is structured in a straight manner . As a result, one measurement of K -sized array is arranged progressively inside the cache. We can transform two-dimensional data matrices into row or column significant one-dimensional matrix in this example. Why do we have to utilise both row and column field of study? Simply to decrease small amount of cach that skip among main storage or cache where as it include each and every similar variables and it able to help us in decrease memory space.

Where as this operation also make the matrix multiplication parallelized by using this beautiful concept of parallel for loops.

3.Effectively using Stack vs. Secondary Storage

Rather than main memory, it is fast access stack ram. Unfortunately, stack memory is restricted. The large input memories were saved in heap memory. We used the stacks with predefined storage resources for quick intermediary computations.

It started 40 threads to do the multiplying process. We divided the work internally in a static way, assuming so each multiplication instruction would take a considerable amount of time. Workflow can be evenly distributed across processes as we are dealing with lengths of 200, 400, 600, 800, 1000, 1200, 1400, 1600, 1800, and 2000.

To prevent race issues, we specifically declared matrix as a common resource in optimization algorithm.

The graph below should help you to understand the growth of performance improvement at every phase.