simpro vtt subproject task 1 final presentation
Post on 03-Jan-2017
261 Views
Preview:
TRANSCRIPT
High-performance computing
in mechanical engineering
SIMPRO VTT subproject, task 1
Janne Keränen, Juha Kortelainen, Marko Antila,
Kai Katajamäki, Aino Manninen, Vesa Nieminen,
Aku Karvinen
2
Task 1.1
Computational resource
management systems
3 13/11/2015 3
Motivation – Why to use computational
resource management
Computational resource management means
Managing concrete computational resources, such as processing and storage
resources and possible peripheral systems, i.e. additional hardware resources
Managing computational queues and computing load balancing, i.e. execution of
computational jobs
Distributed Resource Management Systems1 (DRMS) are meant to
Provide fair access and increase the utilisation rate of the computational resources
Help the users to find the best available resources for their computations
Simplify the submission, executions, monitoring, management, and results retrieval
of large computational cases
The user need not to know the computation hardware, but only give some
requirements for the hardware and DRMS handles the rest
“When the management of the computations becomes too
complicated or is inefficient to be done manually, it is time to
consider using a computational resource management system”
1Or resource management system or resource manager or job scheduler or workload management system …
4 13/11/2015 4
Distributed resource management systems
utilised in Case studies
Grid Engine (Univa Grid Engine, Son of Grid Engine, Open Grid Scheduler) Original developed by Sun Micro Systems, presently the proprietary version is owned by Univa
Open source versions based on the original Sun Grid Engine (SGE):
Son of Grid Engine and Open Grid Scheduler
SLURM (Simple Linux Utility for Resource Management) Development and maintenance coordinated by SchedMD LLC, giving also commercial support
Open source software (GNU GPL v3)
HTCondor Developed and maintained by the University of Wisconsin-Madison, USA
Open source software (Apache License v2.0)
Commercial support available by third party companies (e.g. Red Hat, Inc.)
Techila Developed and maintained by Techila Technologies Oy, Finland
Proprietary software
Some others: Portable Batch System – PBS (PBS Professional, TORQUE)
Grid resource management systems, so-called meta-schedulers (e.g. open source GridWay and Globus Toolkit)
Connect local resource management systems into systems of systems
5 13/11/2015 5
Different needs, different solutions
There are different environments where computational
resource management systems can be used: Distributed, heterogeneous computational resources: e.g. office
networks
Server systems or workstations: one system with one or more
processors and several computational cores
Cluster systems: several computational nodes (computers) with
each one or more processors and several computation cores
Grid systems: network of distributed computational subsystems,
such as clusters and servers (a system of systems)
Different resource management systems are focused
on different needs and concepts E.g. Grid Engine, SLURM, and TORQUE are focused cluster (or
server) systems, and operation is based on computational queues
E.g. Techila and HTCondor are focused on distributed resources
utilising the unused office computer resources
Operation is based on selecting resources for given
computation attributes
None of DRMS can handle all the use scenarios well
Multitude of different workable systems
6 6 13/11/2015
Example case 1: Grid Engine in VTT
computational cluster
Grid Engine (SGE) used to be the most common
DRMS, presently several competing versions
SGE is the DRMS of VTT computation cluster
Close to 2000 cores, Rocks Cluster Distribution,
InfiniBand network, NetApp storage system
Both last Sun version (SGE 6.2u5) and Open Grid
Scheduler 2011.11p1 were tested
SGE cluster is composed of execution machines and
a master machine (and a possible backup master)
Experiences:
The queuing system is not fair and practical for HPC
With default configuration
SGE overloads the nodes problems with HPC
The job scheduling is based on free CPUs, not e.g. free
memory
If memory the limiting resource, challenges with SGE
7 7 13/11/2015
Example case 2: HTCondor with DAKOTA
optimisation software, two scenarios
DAKOTA feeds separate jobs into the
HTCondor computational pool
Each case is executed separately in
the computational pool
Can utilise heterogeneous
independent resources in the pool
Collecting the data for the whole
study requires additional tricks
A DAKOTA job is submitted into the
HTCondor computational pool
The whole DAKOTA study is treated
as one job
Suits best for larger resources in the
computational pool, such as a server
Execution of the DAKOTA run is
straightforward, but retrieving the
results files needs additional tricks
8
Task 1.2
Scripting in high
performance computing
environment
9 13/11/2015 9
Why scripting in HPC?
Automate repeating tasks
Own routines to speed up modelling, analysis,
and post-processing
Interface between different software
Calculation engine in optimisation
Python is presently the most common and best scripting language
Easy to learn and use, includes even classes
Special libraries to many practical needs
NumPy for efficient vector calculation
Matplotlib for plotting
Much more: Web development etc.
Development environments
Python IDLE, Komodo, NetBeans, Pystudio, …
10 13/11/2015 10
Scripting with Python/NumPy: powerful in
vector calculation
Case study: Correlation between two sets of vectors, e.g. mode shape
vectors in dynamics, so-called modal assurance criteria (MAC)
Sizes of vectors can be huge in FEM Vector size tens-of-millions, number of vectors can be several hundred
Direct matrix multiplication (MATLAB or similar) not possible
NumPy library extremely efficient in HPC
Matrix visualisation with Matplotlib.
Easy to couple with optimisation for mode shape identification
To take in use: import numpy import matplotlib
Example output:
Reference mode 10 35.436Hz corresponds to 10 31.838Hz mac value is 0.922056806543
Reference mode 12 39.148Hz corresponds to 11 37.946Hz mac value is 0.924408685671
Reference mode 14 48.975Hz corresponds to 12 42.478Hz mac value is 0.919284572632
11 11 13/11/2015
Python scripting in model updating
DAKOTA parameter file
Apply parameters into template file
Abaqus input file
Write modes to file
MAC calculation with
Write frequencies to file
DAKOTA results file
In model updating, open source
optimisation software DAKOTA was used
Abaqus was used to solve the natural
modes and frequencies
Workflow, run by a Python script:
1. From parameters given by DAKOTA,
Python script creates Abaqus input data
applying parameters to template files
2. The script runs the Abaqus analysis
3. MAC calculation with Python/NumPy
4. The script returns results for DAKOTA
DAKOTA starts a new iteration step
Mathplotlib can be used for visualisation
12
Task 1.3
Multi-physics
simulations for electrical
machine development
13 13 13/11/2015
Elmer open-source FEM: Electrical
machine end-winding model
Electromagnetic fields and forces for end-windings
Parallel computation, partitioning, linear algebra, etc.
Utilises effectively 500 cores, commercial FEM < 20 cores
Elmer, Sisu: Elmer, VTT cluster (Doctor): Commercial FEM, VTT cluster:
14 14 13/11/2015
Multiphysics: Electromagnetics, rotating model, electrical circuits
Comparison of the parallel performance in VTT cluster and CSC
supercomputer Sisu
Elmer can utilise 50-500 cores, depending on the model complexity
Computational time reduces from weeks-days to days-hours
Enables new type of multiphysical modelling:
accurate electromagnetic-thermal and vibro-acoustic analysis
Elmer open-source FEM:
Induction machine model
Partitioning
with rotation:
VTT cluster Sisu
15 13/11/2015 15
Analytical design tool for permanent magnet
electrical machines, for optimisation
1. Choose initial parameters and define rotor dimensions
2. Design armature (stator) winding
3. Define magnet dimensions
4. Define stator tooth and slot dimensions based on target
values of magnetic flux density in stator
5. Calculate machine properties, e.g. shaft power, efficiency,
power factor, losses, temperatures in different parts
Back-EMF is large
enough?
Yes
No
An analytical tool for design of permanent magnet machine was implemented with MATLAB
About 30-40 design variables have to be determined during the design process
Initial parameters: pole pair number, air-gap flux density (T), stator current density (A/m2),
ratio between diameter and height, air-gap (m)
Output: efficiency, mass of magnets, mass of copper, mass of iron
Rotor
Stator
Magnets
Windings
16 16 13/11/2015
Multi-objective optimisation problem for the
electrical machine case
6 objectives
Output power, target 3 MW
Torque density, maximised
Mass, minimised
Efficiency, maximised
Power factor, maximised
Cost, minimised
Constraints:
Slot pitch > 7mm
𝑃𝑜𝑢𝑡
𝑃𝑡𝑎𝑟𝑔𝑒𝑡 should be ≥ 1 and ≤ 1.05
Temperature of permanent
magnets < 100°C
14 input parameters Pole pair number, desired current densities, air-gap
length, magnet width, tangential stress, flux density,
number of slots, stator outer diameter, slot shape, …
Results with DAKOTA: Optimal torque density (Nm/m3) with different
maximum number of function evaluations,
selected values and measures from 50 runs.
500 1000 5000 10000 15000 20000 25000
Best 26,18 47,88 40,36 40,32 49,38 40,32 40,32
Worst 16,48 19,06 21,82 21,82 21,67 21,82 21,82
Average 21,04 25,76 29,59 29,43 29,83 29,37 29,43
Median 20,68 24,88 28,30 28,30 28,14 28,30 28,30
Standard
deviation 2,18 4,77 5,52 5,32 6,13 5,30 5,32
17
Task 1.4
Co-simulation and
parallelisation in technical
simulation
18 13/11/2015 18
Case Study for Fluid-Structure interaction (FSI)
Rotating propeller
Non-rotating cylinder at the
wake represents for example
the body of the azimuthing
thruster
Structural parts surrounded
by a tube, which forms a
cavity for the fluid (water)
Dimensions:
Propeller diameter: 600 mm
Cylinder diameter: 150 mm
Fluid tube diameter: 960 mm
19 13/11/2015 19
FSI Co-Simulation coupling interface: MpCCI (developed at Fraunhofer
Institute SCAI)
So called “weak coupling”: each problem is solved separately and variables
are exchanged before each time step in both directions
During iteration step, data is not changed between the codes
Coupled variables in this FSI are displacements and pressure
Non-conforming meshes; A shape function mapping is used for data
exchange between two non-matching grids
Explicit-transient coupling
Sequential serial coupling algorithm
Time step used: 100 µs
Code A: solid mechanics code (Abaqus)
Code B: CFD code (Fluent)
Fluid-Structure interaction Co-Simulation
process with MpCCI
20 13/11/2015 20
Fluid-Structure interaction Results (1/3)
Totally 8000 time steps was simulated corresponding 4 full rotations of the
propeller
Total co-simulation wall-clock time with workstation was about 80 hours
21 13/11/2015 21
Fluid-Structure interaction Results (2/3)
22 13/11/2015 22
Fluid-Structure interaction Results (3/3)
Axial displacements at tip of the propeller
blade and propeller hub: the blade frequency
dominates
Caused by cylinder structure located at the
wake of the propeller
Transversal and vertical displacements of
the hub: rotating frequency dominates
23
Task 1.5
Large-scale visualisation
and open source tools in
technical computations
24 13/11/2015 24
Open source tools in technical photorealistic
large-scale visualisation
Benefits of open source
Availability of the source code:
Possibility to see what the code does
Possibility to modify the code
Free of license fees (important in HPC)
Continuous software development process,
bugs are corrected (usually) faster security
Tools used:
Salome-platform for pre-processing
snappyHexMesh for grid generation
OpenFOAM for solution
ParaView for post-processing
Blender for rendering
25 13/11/2015 25
Technical photorealistic visualisation:
Example case
Radio-controlled (RC) car
Complex domain
16 million control volumes
Salome OpenFOAM
ParaView Blender
26 13/11/2015 26
Results from RC-car case – Visualisation of the
velocity at the symmetry plane using Paraview
27 13/11/2015 27
Results – Time-averaged streamlines over the
RC car rendered using Blender
28 13/11/2015 28
Surrogate-based optimisation of airfoil using
open source software
Case study: drag minimisation of an airfoil when the minimum lift
is given as an inequality constraint
Optimisation in DAKOTA, calculation of the objective function
using a computational fluid dynamics software OpenFOAM
CFD results produce a non-smooth objective function due to the
numerical errors from the coarseness of the grid
Surrogate-based optimisation methods where the objective
function is replaced by simpler surrogate function
29 29 13/11/2015
Airfoil optimisation results
3
3,5
4
4,5
5
5,5
6
6,5
7
0 1 2 3 4 5 6 7 8
Optimisation Cycle
Design variables
m p
0,012
0,0122
0,0124
0,0126
0,0128
0,013
0,0132
0,0134
0,0136
0,0138
0,014
0 1 2 3 4 5 6 7 8
Optimisation Cycle
Cd
Cd
0,5
0,52
0,54
0,56
0,58
0,6
0,62
0,64
0,66
0,68
0,7
0 1 2 3 4 5 6 7 8
Optimisation Cycle
Cl
Cl
Design variables define the
shape of the airfoil
Cd = drag coefficient
Cl = lift coefficient
(limited to be >0.55)
Optimum (black, initial=red)
TECHNOLOGY FOR BUSINESS
top related