analysis big data by use php with storm
TRANSCRIPT
![Page 1: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/1.jpg)
PHP|
![Page 2: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/2.jpg)
PaaS SAE
Top3 SuperAPP
![Page 3: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/3.jpg)
Pla$orm Services Group
Sam
AWS S3
CDN ATS
Biwing
1k
30+10+
Monika
G o
Rambone
PHP Front-End
Services Panel
Skylee
C++ Go
![Page 4: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/4.jpg)
Content Summary
"
# PHP
PHP
$ %
Case
![Page 5: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/5.jpg)
Real 7me data collec7on
"
![Page 6: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/6.jpg)
How to collect data in real 7me
?
…… ……
![Page 7: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/7.jpg)
Ka<a —— a high-throughput distributed messaging system source from LinkedIn
Producer KaLa LinkedIn Apache
KaLa Producer
&
"
'
(
ModelTwo models: queuing and publish-subscribe. the la<er is more commonly used.
high-level abstracPonEach Topic corresponds to one or more log files. Each Topic has one or more par@@ons.
FeatureScalability & Durability & Reliability & Performance & Fault-tolerant
EcosystemStream processing systems, Hadoop integraPon, monitoring, and deployment tools.
![Page 8: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/8.jpg)
Ka<a should be no7ced when using Ka<a
)
KaLa offsetConsumer
Buffer
KaLa gzip snappy lz4CPU
KaLa KaLa
KaLa
Topic Par@@onPar@@on
Topic Par@@on
ZookeeperKaLa Zookeeper
ZK ZK
![Page 9: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/9.jpg)
Real-7me analysis
"
![Page 10: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/10.jpg)
BackType nathanmarz Storm BackTypeTwiXer Storm TwiXer TwiXer
Apache
' Feature
Storm —— a distributed real-7me computa7on system source from TwiGer
Highly scalable
Fault-tolerant
Guarantees processing
Language agnosPc
* Topology of one Spout
* Topology of three Spout
![Page 11: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/11.jpg)
* UI / Nimbus / ZK / Supervisor
Storm Storm cluster organiza7on structure
Storm UI ZookeeperStorm Nimbus
Storm Supervisor
Topology
![Page 12: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/12.jpg)
Storm Storm mul7 language support
+
Python
PHP
JS
Shell
, Storm use STDIN & STDOUT
support multi language
![Page 13: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/13.jpg)
PHPConnectPool
Storm PHPConnect Storm to PHP
+ PHP
STDIN
STDOUT
+PHP
PHP-FPMPHP
PHP
FastCGI
![Page 14: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/14.jpg)
PHPphp analysis program
PHP
![Page 15: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/15.jpg)
PHPPHP parse a real 7me log
*
-> Kafka -> Storm -> Spout Bolt -> Bolt FCGI
![Page 16: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/16.jpg)
PHPPHP program for parse log
PHP
-
PHP-FPMdynamic PHP
pm.max_children
PHPDB Cache
Storm
web
RD
PHP
Input
PHP
PHP
![Page 17: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/17.jpg)
Storm Bolt PHPStorm bolt send a request by FastCgi protocol
![Page 18: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/18.jpg)
Architecture design
$
![Page 19: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/19.jpg)
Overall architecture design
![Page 20: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/20.jpg)
Performance data
![Page 21: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/21.jpg)
CaseShow Case
%
![Page 22: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/22.jpg)
Sta7s7cal service pla$orm
![Page 23: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/23.jpg)
Sta7s7cal service pla$orm
![Page 24: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/24.jpg)
Real-7me log analyse service
![Page 25: Analysis big data by use php with storm](https://reader034.vdocuments.mx/reader034/viewer/2022052606/58e629151a28ab655d8b4e3f/html5/thumbnails/25.jpg)
Q & A?