Download - ITアーキテクト Vol.22 00.pdf
-
7/28/2019 IT Vol.22 00.pdf
1/121
w w w . i t a r c h i t e c t . j p
IT
1
2 3
22Vol.
Apache Hadoop
-
7/28/2019 IT Vol.22 00.pdf
2/121
22V o l .
024
6
IT
13
Bu s i n e s s M ode l i ng
Commun ica t i on T echn ique
1
128
015
080
092
Ev en t R epo r t
IT 2009 Report
Part 1
O/RTCO
Part 2
Part 3
Web
034
044
056
026
C o n t e n t s
-
7/28/2019 IT Vol.22 00.pdf
3/121
22V o l .
C o n t e n t s
IT
Apache Hadoop
022
085
109
134
086
075
068
065
064
088
096
2
News & Topics
Books
Present
Part 1
HadoopMapReduce/GFS
Part 2
Hadoop
Part 3
Hadoop
3110
-
7/28/2019 IT Vol.22 00.pdf
4/121
-
7/28/2019 IT Vol.22 00.pdf
5/121
0 1 6 IT Vol.22
IT
IT
IT
ITA
IBM
IBM
1
SQL
ITArchitectSummit 2009
R e p o r t
IT
ITA 2009
IT
ITA
ITA
1ROIRet
urn On Investment
ROI
IT
ROI
IT
2
-
7/28/2019 IT Vol.22 00.pdf
6/121
0 1IT Vol.22
SIer
3
SIer
UFJIS
IT
4
1
1
ITA 2009
IT
1 Part 1
UFJ
ITAIT
UFJIS IT
3UFJ
UFJIT
UFJIS3IT
IT
2
IT
UFJIS
1
4
UFJIS400
20019CO2
3IT
UFJIS
2
2
-
7/28/2019 IT Vol.22 00.pdf
7/121
022 IT Vol.22
News &Topics
IBM
IBM2
IBM Computing on De
mand4
1
IBM Computing on Dem
and
CPU2
15
4001CPU 1
150
IT
TOGAF 9
2EA
TOGAFThe Open Group Archi
tecture FrameworkVersion 9
Webhtt
p://www.opengroup.org/togaf/
TOGAF
ADMArchitectu
re Development MethodSOA
IT
Exadata
1
Oracle Exadata
HP
HP Oracle Exad
ata Storage ServerOracle
Database 11g Enterprise Edition
HP Oracle Database
MachineStora
ge ServerOracle Database
2Intel
SAPBusiness Suite 7
SAP2ERP
CRMSCM
SAP Business Suite 7
enhancement package
11
5
MSOSS
1
ASF
Apache Stonehenge
W3COASIS
SOA
MS3
7
MSASF
10
2
KVMKernel-basedVirtual Machine
Linux OSRed Hat Enterprise Li
nux
Red Hat Enterprise Virtu
alization Manager for Servers
Desktops
Red Hat Enter
prise Virtualization Hypervisor
HPSolaris 10HP ProLiant
HP2
HPSolaris 10
HP
HP ProLiantHP BladeSystemSolaris
HP Insight
Solaris 10
HP Integrity
HP-UX
-
7/28/2019 IT Vol.22 00.pdf
8/121
02IT Vol.22 02
IBM4DITA
IBM
2
XML
DITADarwin Information Typi
ng ArchitectureDITA
OASIS
DITA
DITA
DI
TA
1
VMwareMicr
osoft Hyper-V ServerCitrix Xen
Server
Event Calendar
Qcon Tokyo 200949 10
QCon Tokyo 2009
URLhttp://qcontokyo.com/
Pulse Japan 2009415
ibm.com
0120-300-426
URLhttp://www-06.ibm.com/itsolut ions/jp/
solutions/governance/events/pulse/
Agile Japan 2009422
2F
2009
03-5414-5363
URLhttp://www.agilejapan.org/
Oracle OpenWorld Tokyo 2009422 24
URLhttp://www.oracle.co.jp/openworld/2009/
513 15
SODEC
03-3349-8504 FAX03-3349-8500
URLhttp://www.sodec.jp/SODEC/
Interop Tokyo 200968 12
URLhttp://www.interop.jp/
200979 12
03-3349-8507 FAX03-3349-8523
URLhttp://www.digi -fair.jp/
BioFuels World722 24
BioFuels World
03-5800-3534 FAX03-5800-3979
URLhttp://www.biofuels.co .jp/2009/
2009722 24
03-3834-8134 FAX03-3834-8009
E-mailwjc@ric .co.jp
URLhttp://www8.ric .co.jp/expo/wj/
4
5
6
7
SaaS
NTT 3
Webi
ntra-mart WebPlatform Ver.7
SaaSSoftware as a Service
SaaS
SaaS
intra-martintra
-mart
intra-martSaaS
HP
HP 2
HP Insight Dynamics
- VSE 4.1 Orchestration
Recovery
HP SE2120
3
HP
SE2120
Linux
378,000
IBMTeam Concert
IBM2
IBM Rational Team Concert
V.1.0.1.1Re
quirements Composer V1.0
Quality Manag
er V1.0
IBM
Jazz
Team Concert
3
1IT
SIer
IT40
IT
IT
-
7/28/2019 IT Vol.22 00.pdf
9/121
1
024 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
10/121
02IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
11/121
A r c h i t e c t u r e a
PC
1
IT
SIer
Tsutomu Hosokawa
026 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
12/121
-
7/28/2019 IT Vol.22 00.pdf
13/121
A r c h i t e c t u r e a
CPU
1
CPU
123
Web
Web
21
1
2CPUCPU
1
2
CPU...
CPU
CPU
CPU
CPU
1 2
1990
PC
CPU64
PC
028 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
14/121
-
7/28/2019 IT Vol.22 00.pdf
15/121
A r c h i t e c t u r e a
Web
Web
HTTP
Web
Web
1
Web
SQL
Web
JBoss
Cache
Microsoft Velocity
Oracle Coherence
Part 2Coherence
Web
WebWebWeb
030 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
16/121
n d P e r f o r m a n c
1
SQL
Java EE4
2
A
A
001020
ATM ATM
ATM
ATM Web
021040
041060
061080
081100
2 Java EEJDBCAPI
API4Oracle Database
2
03IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
17/121
A r c h i t e c t u r e a
1OS
OSWeb
VMMVirtual Machine Mo
nitor3OS
VMM
2OSOS
4
VMM
OSWeb
5
IT
IT
CPU
1
1
4VMM2
VMM2
WindowsOS UNIX
OS
OS
VMM
Linux
Wi ndo ws U NI X
VMM
Linux
3 VMMOS
032 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
18/121
n d P e r f o r m a n c
1
6
5
Windows UNIX
VMM
UNIX Linux Linux
Windows/ UNIX/Lin uxOS
6
03IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
19/121
A r c h i t e c t u r e a
P a r t 1
IT
1
O / R
T C O
RDBMS
RDBMS
IT
TCOTotal Cost of Ownership
RDBMS
TCO
Yuzo Ishida
P a r t 1
034 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
20/121
n d P e r f o r m a n c
1
1
12
CPU
I/O
21
CPU
CPU
03IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
21/121
A r c h i t e c t u r e a
O / R P a r t 1
CPU
CPU
1
1I/O
RDBMS
CPUI/O
CPU
036 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
22/121
n d P e r f o r m a n c
1
3RDBMS
SQL
CPU
CPU
RDBMSCPU
TCO
CPU
Java
1SQL
SQL
I/O
I/O
3
SQL
2
03IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
23/121
-
7/28/2019 IT Vol.22 00.pdf
24/121
n d P e r f o r m a n c
1
SQLDML
Data Manipulation LanguageDML
DMM
DMM1
DMM1
3
AP
2
AP
I/O
AP
CPU
CPU
DMMData Model Management
2AP
I/O
ID
1
AP
A
B
C
AP
1
AP
ID
1
2
3
4
5
6
1 DMM
DMMDMM
03IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
25/121
A r c h i t e c t u r e a
O / R P a r t 1
DMM
I/O
3
RDBMS
I/O
I/O
I/O
I/O
I/O
I/OI/O
3
1
DMM
3O/R
A
B C
DMM
AP
R/T
040 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
26/121
n d P e r f o r m a n c
1
4
SQL
SQL
CPU
I/OI/
O
I/O
2
2
I/O
CPU
CPU
4
DML)
O/R
AP
AP
DMM
R/T
04IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
27/121
A r c h i t e c t u r e a
O / R P a r t 1
I/OCPU
SQL
SQL
SQL
AP
I/O
5CPU
CPU
R/TAP
5
CPU1 CPU2
CPU
042 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
28/121
n d P e r f o r m a n c
1
2
6
TCO
http://www.itarchitect.jp/O/R
RDBMSCOBOL
I/O
RDBMS
62
1
2
3
4
5
6
201
200
201
2
3
4
1
5
6
200
201
K2
2
3
4
1
5
6
200
201
ID
ID K1 K2
1
2
3
4
5
6
100
201
200
201
04IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
29/121
A r c h i t e c t u r e a
1
Tatsuya Sugi
Fusion MiddlewareFusion Middleware
Tomofumi Nijo
Noriyuki Abe Fusion MiddlewareFusion MiddlewareSC
P a r t 2
044 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
30/121
n d P e r f o r m a n c
1
I/O
ROIReturn On Investment
4
4
RDBMS
RDBMS
RDBMS
ROI
RDBMS
04IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
31/121
A r c h i t e c t u r e a
P a r t 2
2
2
JBoss Cache1
Java Map APIput/get
Java
3
3
1
RDBMS
1RDBMS
RDBMS
RDBMS
SQL
RDBMS
1
API
RDBMS
JDBCODBCSQL
Oracle TimesTenIBM solidDB
put/get
API
memcachedJBoss Cache
put/get
API
1
API
Oracle Cohe renceGemstone GemFi
reGigaSpaces XAP
046 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
32/121
n d P e r f o r m a n c
1
3
2
1
1
2
1
RDBMS
2
Java API
MapReduce
ITWeb 2.0
// Person// CacheFactoryDefaultCacheFactoryCacheFqnNodeJBoss CacheCacheFactory factory = new DefaultCacheFactory();// Cache cache = factory.createCache();
Fqn personData = Fqn.fromString("/person");// PersonNode personNode = cache.getRoot().addChild(personData);
// PersonPerson p1 = new Person(1234, "", "", "");// personNode.put(1234, p1);
// Person// CacheFactoryOracle Coherence
// PersonMap personCache = CacheFactory.getCache("person");
// PersonPerson p1 = new Person(1234, "", "", "");// personCache.put(1234, p1);
1JBoss Cache 3.0
Java Map APIJava
2Oracle Coherence
JBoss Cache
04IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
33/121
A r c h i t e c t u r e a
P a r t 2
21
1
2
CPU
1
1,0001 1,000
1
1
1
get put
A B
C D
2
A B
C D
3
A B
C D
4
A B
C D
048 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
34/121
-
7/28/2019 IT Vol.22 00.pdf
35/121
A r c h i t e c t u r e a
P a r t 2
5
Coherence
MapEvent
100
0.1
Coherence
Invocation
4
Oracle Coherence
2put/get
RDBMS
3
Coherence3
Java
Java
4
// public class ProcessOrderListener implements MapListener {
public void entryInserted(MapEvent event) { /* */ }public void entryUpdated(MapEvent event) {}public void entryDeleted(MapEvent event) {}
}
orderCache.addMapListener(new ProcessOrderListener());
//100Filter filter = new EqualsFilter("getProductId", 100);// Set orders = orderCache.entrySet(filter);
4
5
3
2
1
1
2
1
1
050 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
36/121
n d P e r f o r m a n c
1
3
5
HTTP
2
Coherence
4
1
2 3 4
Invocation
5
05IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
37/121
A r c h i t e c t u r e a
P a r t 2
HTTP
HTTP
1
GC
GC
HTTP
77
GC
JVM
GC
OutOfMemoryError
6
6
052 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
38/121
n d P e r f o r m a n c
1
8
3
2
4
3
HTTP
7HTTP
GC
05IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
39/121
A r c h i t e c t u r e a
P a r t 2
9
1
1
Coherence
8
054 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
40/121
n d P e r f o r m a n c
1
1
10
9
10
Java EE
Java EE
MQ/JMS
Java EE
Java EE
Java EE
MQ/JMS
Java EE
05IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
41/121
A r c h i t e c t u r e a
P a r t 3
Web
Web
24
365
1
2007Web
1
Web
Web
SOA
Service Oriented Architecture
Web1
24 365
Kazuhiko KomabayashiCA
P a r t 3
W e b
1Web1
Typical Revenue Loss per Hour of Do
wntime, 2007
Web 1
650
260
TV 15
TV 113,000
9
89,500
69,000
28,000
ATM 14,500056 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
42/121
n d P e r f o r m a n c
1
SLAService Level Agreement
IT
RDBMS12
COBITCo
ntrol Objectives for Information and related Te
chnology
ITILIT Infrastructure Library
COBIT
ITIL
Web
UNIX
CPU
Web
Web
30
CPU
Web
05IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
43/121
A r c h i t e c t u r e a
W e b P a r t 3
SLA
Web
1
SLA
Web
End-User Experience Management
Web
UNIX
Web
058 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
44/121
n d P e r f o r m a n c
1
CPU
SOA
SOA
SOA
Web
SaaSSoftware as a Servi
cePaaSPlatform as a Service
SOA
05IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
45/121
A r c h i t e c t u r e a
W e b P a r t 3
3
SLA
SLA
SLA
SLASLA
Java
Web
SOA
IT
4
1
1
060 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
46/121
n d P e r f o r m a n c
1
APM
Web
APM
Application Performance Management
APM
Web
CAAPMWily Intr
oscopeWily CEMAPM
APM
CPU
06IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
47/121
A r c h i t e c t u r e a
W e b P a r t 3
Web
SLA
Introscope
APM
APM1
6
1,300
My Circle
24 365Web
APM
APM
SLA
SLA
1
EC1,000
2
1SLA
Introscope
2
Introscope
062 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
48/121
n d P e r f o r m a n c
1
Web
APM
Web
Web
Web
Web
WebLogic ServerWebSphere Application ServerWeb
Oracle DatabaseEAI
17
15,000
Web
Web
APM
9Web
2,000My Circle
APM3
JSP
Web
3APMIntroscope
06IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
49/121
MapReduce
Apache Hadoop
MapReduceGFSGoogle File SystemHadoop
Hadoop
3
Hadoop
Yohei Sasaki
Michihiro Koide
Hiroshi ItouIBM
064 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
50/121
Apache HadoopApache HTTP
ServerTomcatStruts
15
OSS
Core
HadoopHadoop Core
Hadoop1
Hadoop
HadoopWebhttp://hadoop.apache.org/
core/Hadoop
Hadoop
MPIMessage Passing
Interface2
Hadoop
Hadoop
1Hadoop
Hadoop
1
Hadoop
HadoopMapReduce
1
1MapReduce
3Hadoop
HadoopHDFSHadoop Distrib
uted File System
Hadoop
1
Hadoop
Hadoop
Hadoop
1
1
Hadoop
MapReduce/GFS
Part 1
1 HadoopApache Hadoop
Core
Core
2
3 MapReduce
06IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
51/121A ache
Hadoop
MapReduce/GFS
Part 1
CPU
MapReduce
HadoopMapReduce
HDFS
MapReduceGFS
MapReduce
Hadoop
Apache HadoopOSS
1
HadoopMapReduceHDFS
MapReduceGFS
MapReduce/GFS
4
MapReduce
MapReduceMap
Reduce2Map
1
1 ReduceMap
MapReduce
1512
2
1
2Map1
Reduce52
46810
Map
113
4
1Apache HadoopOSS
Apache HadoopOSS
Hadoop CoreMapReduce Framework MapReduce
Hadoop CoreHDFS GFS
ZooKeeper Chubby
hBase BigTable
Hadoop1
1
1
1
1
1
1
1
1
1
1
1
1
HDFS
1Hadoop
066 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
52/121Hadoo
Map3
Reduce
Reduce1
MapReduce
MapReduce
MapReduceSimplifi
ed Data Processing on Large Clusters
MapReduce
MapRedu
ce
GFS
HDFSGFS
GFS
100MBGB
append
GFS3
GFS1
1
64MB
3
246810
1
1Map
Reduce
246810
30
12
1
22
2
32
3
42
4
52
5
2152
06IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
53/121A ache
3MapReduce
MapReduceMapReduce
GFS
Map
MapMap
Hadoop
Part 2
GFS
CPU
Map
CPU
CPU
CPU
MapReduce
3GFS
Hadoop
Hadoop
Hadoop2
Hadoop2
HadoopMapReduce
GFSJavaOSSMapRe
duceGFS2
1MapReduce
Hadoop MapReduce
FrameworkMap/Reduce
Map/ReduceMapReduce
Framework
1HDFS
HDFS
Map/ReduceHDFS
068 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
54/121Hadoo
2
Hadoop 0.19.0
MapReduce Framework
MapReduce Framework
Hadoop
Map/ReduceJobTrack
erTaskTracker2
4JobTrackerTaskTracker
HadoopHad
oop1JobTracker
TaskTracker
2
JobTrackerMapReduce
First come, rst served
TaskTracker
TaskTrackerJobTrackerMap
/ReduceMap
MapReduceReduce
JobTrackerJob
TrackerMapRe
duce FrameworkJobClient
JobClientJobTracker
JobConf
JobTracker
JobConfMap/Reduce
JobClient
JobTracker
MPI
Hadoop
3
mainJobClient
Mapmap
Reducereduce
JobTracker
JobTracker
JobTracker
JobTr ackerHDF S
Map/Reduce
CPU
CPU
CPU
CPU
HDFS
TaskTracker
TaskTracker
TaskTracker
TaskTracker
Map/Reduce
Map/Reduce
Map/Reduce
Map/Reduce
4MapReduce FrameworkJobTrackerTaskTracker
06IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
55/121A ache
MapReduce Framework5
4
Mapmap
CombinerMap
Reduce
ReduceJobConf
Shuffle/SortMap
ReducereduceReduce
MapReduce Framework
MapReduce
HDFS
HDFS
HDFS
MapReduce
MapReduce Framework
6HDFS
NamenodeDatanode HDFS
Namenode
TaskTracker
TaskTracker
TaskTracker
Map ReduceCombiner ShuffleSort
JobTracker
TaskTracker
5MapReduce Framework
Hadoop
Part 2
070 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
56/121Hadoo
DatanodeGFS
HDFS1
GFS
64MBDatanode
Namenode
Datanode
Namenode
HDFS
HDFSMapReduce
HDFSMapReduce
1
HDFS
11
1
HDFS
NamenodeDatanode
HDFS
HDFS
HDFS
HDFS
Hadoop
Hadoop
Namenode
CPU
Datanode
CPU
Datanode
CPU
Datanode
Namenode
6HDFS
07IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
57/121A ache
5
Hadoop3
1JavaHadoop
NamenodeJobTracker
1
Java
MapReduce
OSCent OS 5.2
JavaJDK 6u12HadoopHadoop 0.19.0
localhost
Firewall/SELinuxDisabled
Hadoop3
IPv6Hadoop0.0.0.0
IPv6
IPv6
HadoopHadoop
hadoop
JDK 6.0Hadoop0.19.0
JDK 6.0
Webhttp://java.sun.com/java
se/ja/6/download.html
Hadoop
Hadoop
HadoopWebhttp://hadoop.apache.
org/core/releases.html#DownloadHadoop
TAR
0.19.0tar
/home/hadoop/hadoop-0.19.0
hadoop-0.19.0
confbin
hadoop
HADOOP_HOME
hadoop.bashrc1
export HADOOP_HOME=/home/hadoop/hadoop-0.19.0
$ . .bashrc
HadoopSSH
SSHSSH
SSH
$ ssh-keygen -t rsa
$ cat .ssh/id_rsa.pub >> .ssh/authorized_keys$ chmod 600 .ssh/authorized_keys$ ssh localhost
Hadoop
${HADOOP_HOME}/conf/
hadoop-env.shJAVA_HOME
Java
1JAVA_HOME
hadoop-default.xmlHadoop
hadoop-site.xml
hadoop-default.xml
2
5 Amazon EC2Elastic Computer Cloud
HadoopAMIAmazon Machine ImageOpenSolaris
HadoopLive CD
Hadoop
Hadoop
Part 2
072 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
58/121Hadoo
HDFS
hadoop.tmp.dir/tmp
Linuxtmpwatch
2-
fs.default.namemapred.job.tracker
NamenodeJobTracker
2-dfs.replication
1
12-
mastersNamenode
11
slavesTaskTrackerDatanode
masters11
HDFSNamenode
$ /home/hadoop/hadoop-0.19.0/bin/hadoop namenode -format
Hadoop
$ /home/hadoop/hadoop-0.19.0/bin/start-all.sh
$ /home/hadoop/hadoop-0.19.0/bin/hadoop dfsadmin -report
Webhttp://localhost:50070
HDFS
dfs.http.address50070
JobTraker
$ /home/hadoop/hadoop-0.19.0/bin/hadoop job -list
Webhttp://localhost:50030JobTraker
mapred.job.tracker.http.
address50030
Hadoop
HadoopWikihttp://wiki.apache.org/hadoop/
WordCounthttp://wiki.apache.org/
hadoop/WordCount
WordCount
HadoopWikiWordCount
13MapReduce
3
/home/hadoop/mydomain/
mypkg
6
WordCountWordCount.java
WordCount
JobTrackermain
3JobConfMap/Reduce
# The java implementation to use. Required.export JAVA_HOME=/usr/java/default
1hadoop-env.shJAVA_HOME
6
Webhttp://www.itarchitect.jp/
2hadoop-site.xml
hadoop.tmp.dir/home/${user.name}/hadoop-tmp/
fs.default.namehdfs://localhost:51010
mapred.job.trackerlocalhost:52010
dfs.replication1
07IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
59/121A ache
HDFS
MyMapMyMap.java
MapMapReduceBase
Mapperimplements
4LongWritable
TextTextIntWritab
leMapperGene
rics
MapWordCount
JobConfTextInputFor
mat1
mapmap1
MyReduceMyReduce.java
ReduceMapReduce
BaseReducerimplements
5MyMap
Reducereduce
MyMapmap
reduce
JAR
$ cd /home/hadoop/mysample/mydomain/mypkg/$ javac -classpath /home/hadoop/:/home/hadoop/hadoop-0.19.0/hadoop-0.19.0-core.jar WordCount.java$ cd /home/hadoop/$ jar cvf mysample.jar ./mydomain
HDFS
Hadoophadoop
$ bin/hadoop fs -mkdir /user/hadoop/input$ bin/hadoop fs -copyFromLocal /home/hadoop/messages /user/hadoop/input/
HDFS
hadoop
$ bin/hadoop jar /home/hadoop/mysample.jarmydomain.mypkg.WordCount /user/hadoop/input /user/hadoop/output
public class WordCount {
public static void main(String[] args) throws Exception {JobConf conf = new JobConf(WordCount.class);conf.setJobName("wordcount");conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(IntWritable.class);conf.setMapperClass(MyMap.class);conf.setCombinerClass(MyReduce.class);conf.setReducerClass(MyReduce.class);conf.setInputFormat(TextInputFormat.class);conf.setOutputFormat(TextOutputFormat.class);FileInputFormat.setInputPaths(conf, new Path(args[0]));FileOutputFormat.setOutputPath(conf, new Path(args[1]));JobClient.runJob(conf);
}}
3WordCountWordCount.java
public class MyMap extends MapReduceBase
implements Mapper {private final static IntWritable one = new IntWritable(1);private Text word = new Text();
public void map(LongWritable key, Text value, OutputCollector output, Reporter reporter)throws IOException {
String line = value.toString();StringTokenizer tokenizer = new StringTokenizer(line);while (tokenizer.hasMoreTokens()){
word.set(tokenizer.nextToken());output.collect(word, one);
}}
}
4MyMapMyMap.java
public class MyReduce extends MapReduceBase implements Reducer
{public void reduce(
Text key, Iterator values,OutputCollector output, Reporter reporter)throws IOException {
int sum = 0;while (values.hasNext()){
sum += values.next().get();}
output.collect(key, new IntWritable(sum));}
}
5MyReduceMyReduce.java
Hadoop
Part 2
074 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
60/121
-
7/28/2019 IT Vol.22 00.pdf
61/121A ache
Web
MapReduce
Hadoop
Hadoop
QAQA
HDFS
11QA
JSONJSON
JSON
MapReduce
JSON
Map1QA
{"QAStartDate" : "2004/02/01", "Category" : ["Web", "DataManagement"], }
MapReduce
MapReduce
8Map
Reduce
MapReduce
MapReduce
Hadoop
910
MapReduce
MapReduce
7
7 910
Hadoop
Map Reduce
Map Reduce
Map Reduce
HDFS
QA
QA
QA
QA
Hadoop
7
Hadoop
Part 3
076 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
62/121Hadoo
MapReduce
XenVMware
HadoopXen
VMware
Hadoop
MapReduce
HDFS
2
Hadoop
HDFSNamenode
8Hadoop
HDFS
1HDFS
HDFS
8 NamenodeNamenode
A 120 8 960
B 260 6 1,560
C 300 11 3,300
5,820
Reduce
Map
8MapReduce
1200
1000
800
600
400
200
01 10 20 30 40 50 60 70 80 90 100
2
4
6
8
10
9
1200
1000
800
600
400
200
02 4 86 10
1
10
2030
40
50
60
70
80
90
10 0
10
07IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
63/121A ache
9
HadoopHDFS
TCP/IPHDFS
11Hadoop
Xen/VMwareHa
doop192.
168.128.0/24LANVMware/
Xen192.168.128.0/24
Hadoop
Hadoop
1
Hadoop
11
http://www.forrester.com/Research/Document/
Excerpt/0,7211,47100,00.html
5
1
Part 1Hadoop
Hadoop
9 OS
HDFSHDFSOShadoopdfs
Hadoop Hadoop Hadoop Hadoop
Hadoop
192.168.0.254
hadoop
dfs -cat /path/to/secret
HDFShadoop
192.168.0.0/24
OS
11HDFS
Hadoop
Part 3
078 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
64/121Hadoo
QA
QA
QA
HDFS
10
Hadoop
12
HadoopOSSHBaseApache PigHive
HadoopMapReduce
FrameworkHDFSApa
che HadoopOSS
Hadoop
Wiki
OSSWiki
HBaseBigTable
HDFS
HBaseHBase
HDFS
Apache Pig
Pig LatinePigHadoop
Hive
Hadoop
HiveQLSQL
MapReduceSimplified Data Processing on Large Clust
ersURLhttp://labs.google.com/papers/mapreduce-
osdi04.pdf
The Google File SystemURLhttp://labs.google.
com/papers/gfs-sosp2003.pdf
The Chubby lock service for loosely-coupled distribu
ted systemsURLhttp:// labs.google .com/papers/
chubby-osdi06.pdf
BigtableA Distributed Storage System for Structured
DataURLhttp://labs.google .com/papers/bigtable-
osdi06.pdf
10
SLAService Level Agreement
1
99.920240.00110.48
Hado op
DatanodeTaskTracker Namenode
JobTracker
0.48
Hadoop
12Hadoop
07IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
65/121
080 IT Vol.22
IT
IT
IT
Takashi Arai
11
2,500
2
1991
-
7/28/2019 IT Vol.22 00.pdf
66/121
08IT Vol.22
2
1
2
1
1
13
1
3
1
-
7/28/2019 IT Vol.22 00.pdf
67/121
082 IT Vol.22
1
4
2
SCMSupply Chain Management
2
1
1
-
7/28/2019 IT Vol.22 00.pdf
68/121
08IT Vol.22
1
Everio GZ-MG330
2
Everio GZ-
MG3305117
2
Everio GZ-MG330
2
3
2
2
-
7/28/2019 IT Vol.22 00.pdf
69/121
084 IT Vol.22
3-3
IT
IT
IT
IC
IT
IT
IT
3
10
1
4
33IT
100
3
4
3
100
-
7/28/2019 IT Vol.22 00.pdf
70/121
08
Books
IT
1,575
IT
1714
57
IT
James Shore
3,780
1
XP1
2,310
7
882
1
Release It!
Michael T. Nygard
3,780
IT
IT Vol.22
JavaScott W. Ambler
1,995
Java
-
7/28/2019 IT Vol.22 00.pdf
71/121
086
Hiroaki Oizumi
IT
1990
SCMEAISOA
1985
UNIX
X Wi
ndow System
1990
SCM
IT
SCM
2004
EAI1
ERP
EAI
SOAESBEnterpr
ise Service Bus
ERPRDBMSWeb 2.0
SOAWeb 2.0
SOABPMB
usiness Process Management
IT
IT
IT
IT
IT
IT
21Vol.
P e r s o n a l H i s t o r y o f T o p A r c h i t e c t
Masanori SakaguchiKOYO
IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
72/121
08IT Vol.03
IT
1959
1985
1993
UNIXC/S
1997
1999 SCM
2002
2004 EAI
2006 SOA
SOACRM
2009 SOA
-
7/28/2019 IT Vol.22 00.pdf
73/121
IT
088 IT Vol.22
To Do
100
1
To
Do
To Do
To Do
To Do
2
To Do
Outlook
3
To Do
30
To Do
4
5
David Allen
David Allen
1 1
2
3 OutlookTo Do
4 1
5
1PC
IT
IT
Norio kasahara
Vol.20
-
7/28/2019 IT Vol.22 00.pdf
74/121
08IT Vol.22
IT
67
4
8PDA
PC
PC
9
4
PDA
PC
11
IT
10
11
112
5
6
7
8 Gmail
9
10 1
11
12
A
n
a
l
y
z
i
n
g
t
h
e
a
r
c
h
i
t
e
c
t
u
re
f
r
e
e
l
y
1
2
2
-
7/28/2019 IT Vol.22 00.pdf
75/121
090 IT Vol.22
13
2
Allen
5
14
FAX
5
15
13
14
15 1
2
GPS
LT
10km
LSDLong Slow Distance
3
-
7/28/2019 IT Vol.22 00.pdf
76/121
09IT Vol.22
16
17
18
1
1
19
2
202
Mandal-Arthttp://www.ma
ndal-art .com/9
8
3
1
09
16 17
18 Al len
19 Al len
20
IT
Vol.
To DoTo Be
3
-
7/28/2019 IT Vol.22 00.pdf
77/121
-
7/28/2019 IT Vol.22 00.pdf
78/121
-
7/28/2019 IT Vol.22 00.pdf
79/121
-
7/28/2019 IT Vol.22 00.pdf
80/121
-
7/28/2019 IT Vol.22 00.pdf
81/121
2
096 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
82/121
IT
IT
IT
MALT
Koichi Hayashi
09IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
83/121
So
What/Why SoMECE
3
1
So What/Why So
So What/Why SoSo
What
Why So
2
MECE
MECE
MECEMutually Exclusive and Collectively
Exhaustive
MECE
13
So What/Why So
MECE
098 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
84/121
2
MECE
1
1MECE
3
1
IT
MALTMo
deling As Logical Thinking2
IT
MALT2
1IT
IT
Modeling
1
3C
PDCA
09IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
85/121
1
MALT
MALTIT
MALTIT
ERUML
MALT
2
1100ml401
100ml
100100
200ml
2
2MALTIT
MALT
So What/Why So
MECE
IT
100 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
86/121
2
1
111
23
2040020
40/1001000/100100/100100
20
IT
3
IT
MALT
7
MALT7
4
3
IT
10IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
87/121
3
IT
MVC
MALT
4MALT7
MALT
102 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
88/121
2
7FIRE
AIM
4
DQQDocument Quality Quad
W
7
MALT
AIMFIRE
1
Web
5
66-
6-
Web
6-
5Web
Web
Why So
So What
10IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
89/121
3
AIMFIRE
MALTA
IMFIRE
FIREAI
M2
FIRE
FIRE
FactInterpretation
REsponse3
FIRE
73
3
Web
3 1980Donald A. Norman
7
7
6
Web
Web
104 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
90/121
2
86-
6-
6-
6-
6-
Web
AIM
MALT
AIM9Ad
vanced Interpretation Matching
AIM2
7FIRE
Interpretation
Fact
REesponse
8
Web
9AIM
10IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
91/121
AIM
Web
AIM
Web10
Web
3Web
Web
2
10-
2
1
10Web
Web
Web
Web
106 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
92/121
2
MAL
TWeb
10
10-
Web
WebWeb
11
2
10-
Web
1111-
2
1
MALT
1
1
11-
1
11-
10IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
93/121
7MALT
MALT
IT
MALT
5IDG
IT
6IT
MALT
5
IT
IT
Web5
IT
IT
2,000
2009524
11
Web
108 IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
94/121
10
P resent
12
2009424
11
Vol.232009525
3
19cm
5cm
1
4
85
1
IT
5
85
Java2
Java
1
6
85
1
IDG
1
2
IDG
1
1
IT Vol.22
Vol .21
DSi
USB
IT
2
1
4
4
2
Java
2
IDG
1
PLAY Mou
se1
PC
OSWindo
ws 2000/XP/VistaMac
OS X 10.3.9
-
7/28/2019 IT Vol.22 00.pdf
95/121
100IT
1
3
NoriakiOkui
3
IT Vol.22110
-
7/28/2019 IT Vol.22 00.pdf
96/121
11IT Vol.22
1
PM
IT
1IT
http://www.gartner.co.jp/b3i/analy
st/090107/2003 6
IT
IT
200711IT
6
4IT
IT
-
7/28/2019 IT Vol.22 00.pdf
97/121
112 IT Vol.22
IT
PM
IT
NHK
2007
-
7/28/2019 IT Vol.22 00.pdf
98/121
3
11IT Vol.22
3
3
1
3
13
3
-
7/28/2019 IT Vol.22 00.pdf
99/121
114 IT Vol.22
3
-
7/28/2019 IT Vol.22 00.pdf
100/121
3
11IT Vol.22
-
7/28/2019 IT Vol.22 00.pdf
101/121
116 IT Vol.22
IT
1
2
1
1
2
-
7/28/2019 IT Vol.22 00.pdf
102/121
3
11IT Vol.22
3
1
1
-
7/28/2019 IT Vol.22 00.pdf
103/121
118 IT Vol.22
IT
IT
3
1
1
-
7/28/2019 IT Vol.22 00.pdf
104/121
3
11IT Vol.22
1
11
-
7/28/2019 IT Vol.22 00.pdf
105/121
120 IT Vol.22
1
5BMW400NHK
-
7/28/2019 IT Vol.22 00.pdf
106/121
3
12IT Vol.22
3
2
2
3
-
7/28/2019 IT Vol.22 00.pdf
107/121
122 IT Vol.22
5
3
IT
4
4
5
-
7/28/2019 IT Vol.22 00.pdf
108/121
3
12IT Vol.22
2
AB
A
A
SWOT
3
SWOT
SWOTSStrengthsWWeaknes
sesOOpportunitiesTThreats30
A
A
SWOT2
3
A
SWOT
4
2A
SStrengths WWeaknesses
3A
OOpportunities TThreats
4
S W
O
T
-
7/28/2019 IT Vol.22 00.pdf
109/121
124 IT Vol.22
BC
A
BC
4
4
A
4
5
3
A
5
2
A
B
C
A
B
C
A
B
C
A
C
A:
B
C
A
B
C
A
B
C
A
B
C
A
B
C
-
7/28/2019 IT Vol.22 00.pdf
110/121
3
12IT Vol.22
3
A
B
C
A
B
C
A
B
C
A
B
C
A
B
C
A
B
C
A
B
C
A
B
C
A
B
C
A
B
C
A
B
C
A
B
C
4
A
B
C
A
B
C
AB
C
A
B
C
A
B
C
A
B
C
A
B
C
-
7/28/2019 IT Vol.22 00.pdf
111/121
126 IT Vol.22
6
6
-
7/28/2019 IT Vol.22 00.pdf
112/121
3
12IT Vol.22
1
1
1
-
7/28/2019 IT Vol.22 00.pdf
113/121
ITIT!!
128 IT Vol.22
Satoshi Akeniwa
6
1
10
1
BPM
N
6
ITBPMS
Business Process Management Suites
BPMS IT
6
-
7/28/2019 IT Vol.22 00.pdf
114/121
12IT Vol.22
1
1
3
1
1
As IsTo Be2
1
2
2
3
BPMS
2
1
1
-
7/28/2019 IT Vol.22 00.pdf
115/121
130 IT Vol.22
6
1
3
32
1
10
10
21
2
2
1
1
2
2
3
-
7/28/2019 IT Vol.22 00.pdf
116/121
13IT Vol.22
3
3
31
2
1
5
4
4
1
3
2
52
3
1
2
1
2
4
5
-
7/28/2019 IT Vol.22 00.pdf
117/121
132 IT Vol.22
6
65W1H
5W1H
2
2
1
2
4
1015 1825
15401330
13301035
15401035
810
5
1
1
-
7/28/2019 IT Vol.22 00.pdf
118/121
-
7/28/2019 IT Vol.22 00.pdf
119/121
A u t h o r ' s P r o f i l e
134 IT Vol.22
SI
RDBORDB
P.056
IBM
5
P.110
2001
Cohere
nce
P.044
XML WebSOA
IT
P.096
Java/SOA
WebLogicApplication Grid
Japan OSOA
CommunityJava
P.044
2000
Java/C++Web
2006
Coheren
ce/WebLogic
P.044
19931999
IT
2001
MBA
P.034
20
CIO
P.026
-
7/28/2019 IT Vol.22 00.pdf
120/121
13IT Vol.22
2002IBM
1990
LinuxWeb
P.064
Unifed Process
IT
IBM
2004
2006Web
P.064
IBM
2001
AIXHACMP
2003
P.064
SE
90 SE
BP
P.092
P.088
P.080
SI
BPMN
P.128
-
7/28/2019 IT Vol.22 00.pdf
121/121
1
2
ITIT
Vol.232009525
IDG
N e x t
I s s u e
S t a f f