DeapSECURE module 6: Parallel and High-Performance Programming

Motivation for parallel programming: too big or too long

Parallel computing definition: using multiple computing resources to solve a bigger computational problem

Loosely and tightly coupled problems

Key components of tightly coupled parallel programming: problem decomposition, concurrency, communication and coordination

Serial computer’s major parts: CPU, memory, storage, Input/Output.

Serial programming: one process per cpu in sequential order.

Parallel computer: connected multiple computers that can coordinate and communicate.

Shared memory: utilizes multi-core threading (within one node).

Distributed memory: utilizes explicit message passing to exchange information between processors.

Processes can exange information using MPI4PY constructs.

Messages can be any object types.

Commuincations can be peer to peer (comm.send and comm.recv) or collective (comm.gather, comm.scatter…).

An MPI4PY COMM_WORLD object is necessary to manage operations between processes.

Use of if (rank == SOMETHING) ... else construct to make processes execute different tasks.

Rank 0’s special role.

A message can be any object type.

There are different types of communication types.

Problem decomposition is problem dependent.

Utilize problem decomposition to boost parallel efficiency. Sometimes this requires irregular partitions.

Provide a commonly occurring template for a parallel (computational) program.

Identify additional pieces in a code as a result of parallelization.

Provide a common pattern & workflow of parallelization of a program.

Parallel code speedup analysis and why ideal cannot be achieved.

Be able to identify sections of code to improve to reduce runtime.

The priority is to use compiled-language routines/libraries instead of pure Python, since they will reduce runtime.

You can use phe.paillier Python module to encrypt and decrypt images using Paillier Homomorphic Encryption.

Utilize problem decomposition to parallelize, but do not allow chunks of data to overlap

Utilizing the same key among all processes will allow decrypting by one process.

FIXME