On this page
In the late 1980s India wanted a supercomputer for work such as weather forecasting and could not buy the one it needed. The answer was to build one. The Centre for Development of Advanced Computing (C-DAC) in Pune was set up to do it. Vijay Bhatkar, whom C-DAC lists as its former executive director, described the result, the PARAM supercomputer, to the world's parallel computing researchers in 1994. This post covers what C-DAC set out to build, why it chose many processors working together, a worked example on processor speed and Amdahl's law, and what students can learn from the project.
Why did India build its own supercomputer?
C-DAC's About Us page says that the setting up of C-DAC in 1988 "itself was to built Supercomputers in context of denial of import of Supercomputers by USA", and that it has since built several generations of supercomputer "starting from PARAM with 1 GF in 1988".
C-DAC's testimonials page reproduces a speech by R. A. Mashelkar, then Director General of the Council of Scientific and Industrial Research. He recalled that a Cray X-MP "was something that we needed for weather forecasting" and was not available, so Indian scientists "met the challenge by using massively parallel processing computing technology to create a supercomputer". In his words, PARAM was delivered in less than the three years C-DAC was given, and for less than the 10 million dollars it was given.
What was C-DAC's first mission?
Bhatkar described it himself in a 1994 paper, PARAM parallel supercomputer: architecture, programming environment, and applications, at the International Parallel Processing Symposium. Its abstract says C-DAC was launched by the Government of India as a national initiative "with a first 3-year mission" to design and bring into commercial use a parallel supercomputer "with peak performance exceeding 1 GFLOPS", together with storage, a parallel programming environment, and application programs of national importance.
The abstract calls parallel processing a "leap-frog path for supercomputing". It reports that the mission took 300 man-years of effort and delivered the PARAM supercomputer, the programming environment PARAS and a range of science and engineering applications. It also previews a second mission: a teraflops machine by 1997-98.
The C-DAC testimonials page lists Bhatkar as "Former Executive Director, C-DAC". The Government of India honoured him with the Padma Shri in 2000, listed as Dr. Vijay Pandurang Bhatkar, Science and Engineering (Information Technology), Pune, and the Padma Bhushan in 2015.
Why use many processors instead of one fast one?
One way to build a supercomputer is around a few very fast processors. A parallel machine instead joins many processors and splits the work between them. The catch is that not all work can be split.
In 1967 Gene Amdahl examined the claim, as his abstract puts it, that real advances could be made "only by interconnection of a multiplicity of computers", in Validity of the single processor approach to achieving large scale computing capabilities. His point, now called Amdahl's law, is that the part of a job that must run step by step limits how much extra processors can help. It is usually written:
speedup S = 1 ÷ ((1 − p) + p ÷ N)
where p is the fraction of the work that can run in parallel and N is the number of processors.
Worked example: GFLOPS and Amdahl's law
1 GFLOPS means 10^9 floating-point operations (such as multiplying two decimal numbers) per second. These are sample numbers for practice.
A big job. A forecast needs 3.6 × 10^12 operations. At 1 GFLOPS: 3.6 × 10^12 ÷ 10^9 = 3,600 s = 1 hour.
A slower machine. At 10^6 operations per second, the same job takes 3.6 × 10^6 s. Divide by 86,400 s per day: about 41.7 days. A forecast that arrives after the weather is useless.
Amdahl with 64 processors. Suppose 95% of the job can run in parallel, so p = 0.95 and N = 64. Then p ÷ N = 0.95 ÷ 64 ≈ 0.0148. Add 1 − p = 0.05 to get 0.0648. S = 1 ÷ 0.0648 ≈ 15.4.
The ceiling. As N grows very large, p ÷ N goes to zero and S approaches 1 ÷ 0.05 = 20. No number of processors can make this job more than 20 times faster.
Common mistakes:
Assuming 64 processors give 64 times the speed. Only the parallel part speeds up.
Mixing up p and 1 − p. Write down which fraction is serial before you start.
Ignoring communication. Real processors spend time exchanging data, so measured speedup is usually lower still.
This is why the PARAM mission included a programming environment and applications, not just hardware. A fast machine is only useful when programs can keep its processors busy.
What came after PARAM?
C-DAC's About page describes "multiple generations" of PARAM machines since 1988. C-DAC's testimonials page notes that its terascale PARAM Padma entered the Top500 list of the world's fastest computers at rank 171, which it describes as the first time an Indian supercomputer made the list. Our profile of Narendra Karmarkar covers another Indian computer scientist whose work rests on making big calculations fast.
What can a student take from this project?
A constraint can set the direction. Denied one kind of machine, C-DAC chose a different architecture rather than a copy.
Set a measurable target. "Peak performance exceeding 1 GFLOPS" in three years was a goal anyone could check.
Find the bottleneck first. Amdahl's law says the serial part limits you. In study too, the weakest topic often limits your score more than polishing a strong one.
If you are not sure which topic is your bottleneck, Fermi's AI tutor works out what you know and builds a plan for the session.
Next step: redo step 3 with p = 0.99 and N = 64, then find the ceiling, and see how much the serial 1% costs.
