Артём Акуляков - f# for data analysis
TRANSCRIPT
![Page 1: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/1.jpg)
f# for data analysisАртём Акуляков
![Page 2: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/2.jpg)
я- dotnet, python, js, go- ( ), data analysis, ml- fintech стартап, senior engineer
![Page 3: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/3.jpg)
data analysis
![Page 4: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/4.jpg)
data analysis- math and computer science- состоит
- очистка- трансформация- дополнение- фильтрация- моделирование- кластеризация- поиск корреляций- проверка гипотез- ...
- … rocket sience
![Page 5: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/5.jpg)
data analysis не всегда rocket sience- проект на поддержку без описания структуры данных- расследование по логам- сложная аналитика- ...
![Page 6: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/6.jpg)
житейскийdata analysis
![Page 7: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/7.jpg)
f#
![Page 8: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/8.jpg)
f# разработчики самые счастливые
![Page 9: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/9.jpg)
f#let rec map func lst =
match lst with
| [] -> []
| head :: tail -> func head :: map func tail
let myList = [1;3;5]
let newList = map (fun x -> x + 1) myList
![Page 10: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/10.jpg)
f#
- functional-first programming language- компилируемый & интерпретируемый- dotnet- linux, osx, win, +- монады, матан и вся страшная жесть- хорош для data analysis
![Page 11: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/11.jpg)
data analysis?
![Page 12: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/12.jpg)
data analysis1. доступ к данным2. визуализация3. манипулирование данными4. интеграция(?)
![Page 13: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/13.jpg)
1 доступ к данным
![Page 14: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/14.jpg)
1 доступ к даннымFSharp.Data & FSharp.Data.TypeProviders
- sql db- web & files
- json- xml- csv- html
- world bank- twitter- ...
![Page 15: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/15.jpg)
FSharp.Dataopen FSharp.Data
type GitHub = JsonProvider<"https://api.github.com/repos/dotnet/coreclr/issues">
let topRecentlyUpdatedIssues =
GitHub.GetSamples()
|> Seq.sortBy (fun x -> x.UpdatedAt)
|> Seq.truncate 5
![Page 16: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/16.jpg)
FSharp.Dataopen FSharp.Data
type GitHub = JsonProvider<"https://api.github.com/repos/dotnet/coreclr/issues">
let topRecentlyUpdatedIssues =
GitHub.GetSamples()
|> Seq.sortBy (fun x -> x.UpdatedAt)
|> Seq.truncate 5
![Page 17: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/17.jpg)
FSharp.Dataopen FSharp.Data
type GitHub = JsonProvider<"https://api.github.com/repos/dotnet/coreclr/issues">
let topRecentlyUpdatedIssues =
GitHub.GetSamples()
|> Seq.sortBy (fun x -> x.UpdatedAt)
|> Seq.truncate 5
![Page 18: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/18.jpg)
FSharp.Data
![Page 19: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/19.jpg)
2 визуализация
![Page 20: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/20.jpg)
2 визуализация- xplot- FSharp.Charting
![Page 21: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/21.jpg)
FSharp.Chartingopen FSharp.Charting
open System
let d1 = [for x in 0 .. 100 -> (x, 1.0 / (float x + 1.) )]
let d2 = [for x in 0 .. 100 -> (x, Math.Sin(float(x)))]
Chart.Rows [
Chart.Line(d1,Name="d1",Title="d1")
Chart.Column(d2,Name="d2",Title="d2")
]
![Page 22: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/22.jpg)
FSharp.Chartingopen FSharp.Charting
open System
let d1 = [for x in 0 .. 100 -> (x, 1.0 / (float x + 1.) )]
let d2 = [for x in 0 .. 100 -> (x, Math.Sin(float(x)))]
Chart.Rows [
Chart.Line(d1,Name="d1",Title="d1")
Chart.Column(d2,Name="d2",Title="d2")
]
![Page 23: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/23.jpg)
FSharp.Chartingopen FSharp.Charting
open System
let d1 = [for x in 0 .. 100 -> (x, 1.0 / (float x + 1.) )]
let d2 = [for x in 0 .. 100 -> (x, Math.Sin(float(x)))]
Chart.Rows [
Chart.Line(d1,Name="d1",Title="d1")
Chart.Column(d2,Name="d2",Title="d2")
]
![Page 24: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/24.jpg)
FSharp.Charting
![Page 25: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/25.jpg)
FSharp.Chartingopen FSharp.Charting
open System
let d1 = [for x in 0 .. 100 -> (x, 1.0 / (float x + 1.) )]
let d2 = [for x in 0 .. 100 -> (x, Math.Sin(float(x)))]
Chart.Combine [
Chart.Line(d1,Name="d1",Title="d1")
Chart.Column(d2,Name="d2",Title="d2")
]
![Page 26: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/26.jpg)
FSharp.Charting
![Page 27: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/27.jpg)
3 манипулирование данными
![Page 28: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/28.jpg)
3 манипулирование данными- repl & ( )
![Page 29: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/29.jpg)
3 манипулирование данными- repl & ( )- Deedle
![Page 30: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/30.jpg)
Deedle- аналог pandas из мира python- недо-Excel на стероидах- приправа из статистики
![Page 31: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/31.jpg)
Deedle- Series- Frames- Stats
![Page 32: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/32.jpg)
Deedleopen FSharp.Data
let WorldBank = WorldBankData.GetDataContext()
let co2Indicator =
WorldBank
.Countries.``Russian Federation``
.Indicators.``CO2 emissions (metric tons per capita)``
let populationIndicator =
WorldBank
.Countries.``Russian Federation``
.Indicators.``Population, total``
![Page 33: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/33.jpg)
Deedleopen Deedle
open FSharp.Data
let WorldBank = ...
let co2Series = co2Indicator |> Series.ofObservations
let populationSeries = populationIndicator |> Series.ofObservations
let frame = Frame(["co2";"population"],[co2Series;populationSeries])|>Frame.dropSparseRows
frame?totalCo2 <- frame?co2 * frame?population
let avarageCo2 = frame?totalCo2 |> Stats.median
![Page 34: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/34.jpg)
Deedleopen Deedle
open FSharp.Data
let WorldBank = ...
let co2Series = co2Indicator |> Series.ofObservations
let populationSeries = populationIndicator |> Series.ofObservations
let frame = Frame(["co2";"population"],[co2Series;populationSeries])|>Frame.dropSparseRows
frame?totalCo2 <- frame?co2 * frame?population
let avarageCo2 = frame?totalCo2 |> Stats.median
![Page 35: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/35.jpg)
Deedleopen Deedle
open FSharp.Data
let WorldBank = ...
let co2Series = co2Indicator |> Series.ofObservations
let populationSeries = populationIndicator |> Series.ofObservations
let frame = Frame(["co2";"population"],[co2Series;populationSeries])|>Frame.dropSparseRows
frame?totalCo2 <- frame?co2 * frame?population
let avarageCo2 = frame?totalCo2 |> Stats.median
![Page 36: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/36.jpg)
Deedle
![Page 37: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/37.jpg)
Deedleopen Deedle
open FSharp.Data
let WorldBank = ...
let co2Series = co2Indicator |> Series.ofObservations
let populationSeries = populationIndicator |> Series.ofObservations
let frame = Frame(["co2";"population"],[co2Series;populationSeries])|>Frame.dropSparseRows
frame?totalCo2 <- frame?co2 * frame?population
let avarageCo2 = frame?totalCo2 |> Stats.median
![Page 38: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/38.jpg)
Deedleopen Deedle
open FSharp.Data
let WorldBank = ...
let co2Series = co2Indicator |> Series.ofObservations
let populationSeries = populationIndicator |> Series.ofObservations
let frame = Frame(["co2";"population"],[co2Series;populationSeries])|>Frame.dropSparseRows
frame?totalCo2 <- frame?co2 * frame?population
let avarageCo2 = frame?totalCo2 |> Stats.median
![Page 39: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/39.jpg)
Deedleopen Deedle
open FSharp.Data
let WorldBank = ...
let co2Series = co2Indicator |> Series.ofObservations
let populationSeries = populationIndicator |> Series.ofObservations
let frame = Frame(["co2";"population"],[co2Series;populationSeries])|>Frame.dropSparseRows
frame?totalCo2 <- frame?co2 * frame?population
let avarageCo2 = frame?totalCo2 |> Stats.median
![Page 40: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/40.jpg)
3 манипулирование данными- repl & ( )- Deedle- Math.Net
![Page 41: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/41.jpg)
Math.Net- матричные вычисления- статистика- решение систем уравнений- регрессия- ...
![Page 42: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/42.jpg)
Math.Netopen MathNet.Numerics.Statistics
...
let co2Values = frame?co2 |> Series.values
let populationValues = frame?population |> Series.values
let coef = Correlation.Spearman(populationValues, co2Values)
![Page 43: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/43.jpg)
Math.Netopen MathNet.Numerics.Statistics
...
let co2Values = frame?co2 |> Series.values
let populationValues = frame?population |> Series.values
let coef = Correlation.Spearman(populationValues, co2Values)
![Page 44: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/44.jpg)
Math.Netopen MathNet.Numerics.Statistics
...
let co2Values = frame?co2 |> Series.values
let populationValues = frame?population |> Series.values
let coef = Correlation.Spearman(populationValues, co2Values)
![Page 45: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/45.jpg)
Math.Netopen MathNet.Numerics.Statistics
...
let co2Values = frame?co2 |> Series.values
let populationValues = frame?population |> Series.values
let coef = Correlation.Spearman(populationValues, co2Values)
?> -0.2756916996
![Page 46: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/46.jpg)
3 манипулирование данными- repl & ( )- Deedle- Math.Net- R Provider
![Page 47: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/47.jpg)
R Provider- удобно получать и обрабатывать данные с f#- удобно делать расчеты и визуализацию с R
![Page 48: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/48.jpg)
R Provider...
open RDotNet
open RProvider
let rng = Random()
let rand () = rng.NextDouble()
let X1s = [ for i in 0 .. 9 -> 10. * rand () ]
let X2s = [ for i in 0 .. 9 -> 5. * rand () ]
let Ys = [ for i in 0 .. 9 -> 5. + 3. * X1s.[i] - 2. * X2s.[i] + rand () ]
R.plot(Ys)
![Page 49: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/49.jpg)
R Provider...
open RDotNet
open RProvider
let rng = Random()
let rand () = rng.NextDouble()
let X1s = [ for i in 0 .. 9 -> 10. * rand () ]
let X2s = [ for i in 0 .. 9 -> 5. * rand () ]
let Ys = [ for i in 0 .. 9 -> 5. + 3. * X1s.[i] - 2. * X2s.[i] + rand () ]
R.plot(Ys)
![Page 50: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/50.jpg)
R Provider...
open RDotNet
open RProvider
let rng = Random()
let rand () = rng.NextDouble()
let X1s = [ for i in 0 .. 9 -> 10. * rand () ]
let X2s = [ for i in 0 .. 9 -> 5. * rand () ]
let Ys = [ for i in 0 .. 9 -> 5. + 3. * X1s.[i] - 2. * X2s.[i] + rand () ]
R.plot(Ys)
![Page 51: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/51.jpg)
R Provider...
open RDotNet
open RProvider
let rng = Random()
let rand () = rng.NextDouble()
let X1s = [ for i in 0 .. 9 -> 10. * rand () ]
let X2s = [ for i in 0 .. 9 -> 5. * rand () ]
let Ys = [ for i in 0 .. 9 -> 5. + 3. * X1s.[i] - 2. * X2s.[i] + rand () ]
R.plot(Ys)
![Page 52: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/52.jpg)
R Provider
![Page 53: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/53.jpg)
R Provider...
let ds = namedParams ["Y", box Ys;"X1", box X1s;"X2", box X2s;] |> R.data_frame
let result = R.lm(formula = "Y~X1+X2", data = ds)
![Page 54: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/54.jpg)
R Provider...
let ds = namedParams ["Y", box Ys;"X1", box X1s;"X2", box X2s;] |> R.data_frame
let result = R.lm(formula = "Y~X1+X2", data = ds)
![Page 55: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/55.jpg)
R Provider...
let ds = namedParams ["Y", box Ys;"X1", box X1s;"X2", box X2s;] |> R.data_frame
let result = R.lm(formula = "Y~X1+X2", data = ds)
![Page 56: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/56.jpg)
R Provider...
let ds = namedParams ["Y", box Ys;"X1", box X1s;"X2", box X2s;] |> R.data_frame
let result = R.lm(formula = "Y~X1+X2", data = ds)
?> Coefficients:
(Intercept) X1 X2
5.444 2.988 -1.926
![Page 57: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/57.jpg)
R Provider...
let Ys = [ for i in 0 .. 9 -> 5. + 3. * X1s.[i] - 2. * X2s.[i] + rand () ]
...
?> Coefficients:
(Intercept) X1 X2
5.444 2.988 -1.926
![Page 58: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/58.jpg)
3 манипулирование данными- repl & ( )- Deedle- Math.Net- R Provider- {m}brace
![Page 59: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/59.jpg)
{m}brace- azure- aws(?)
![Page 60: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/60.jpg)
{m}braceopen MBrace.Core
...
let sourceValues = [|for x in 0 .. 16 -> [|for y in 0 .. 2000 -> rand()|]|]
let cluster = ThespianCluster.InitOnCurrentMachine(4, ...)
let r =
[|for x in sourceValues -> cloud { return x |> Array.sum}|]
|> Cloud.Parallel
|> cluster.Run
|> Array.sum
cluster.KillAllWorkers()
![Page 61: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/61.jpg)
{m}braceopen MBrace.Core
...
let sourceValues = [|for x in 0 .. 16 -> [|for y in 0 .. 2000 -> rand()|]|]
let cluster = ThespianCluster.InitOnCurrentMachine(4, ...)
let r =
[|for x in sourceValues -> cloud { return x |> Array.sum}|]
|> Cloud.Parallel
|> cluster.Run
|> Array.sum
cluster.KillAllWorkers()
![Page 62: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/62.jpg)
{m}braceopen MBrace.Core
...
let sourceValues = [|for x in 0 .. 16 -> [|for y in 0 .. 2000 -> rand()|]|]
let cluster = ThespianCluster.InitOnCurrentMachine(4, ...)
let r =
[|for x in sourceValues -> cloud { return x |> Array.sum}|]
|> Cloud.Parallel
|> cluster.Run
|> Array.sum
cluster.KillAllWorkers()
![Page 63: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/63.jpg)
{m}braceopen MBrace.Core
...
let sourceValues = [|for x in 0 .. 16 -> [|for y in 0 .. 2000 -> rand()|]|]
let cluster = ThespianCluster.InitOnCurrentMachine(4, ...)
let r =
[|for x in sourceValues -> cloud { return x |> Array.sum}|]
|> Cloud.Parallel
|> cluster.Run
|> Array.sum
cluster.KillAllWorkers()
![Page 64: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/64.jpg)
{m}braceopen MBrace.Core
...
let sourceValues = [|for x in 0 .. 16 -> [|for y in 0 .. 2000 -> rand()|]|]
let cluster = ThespianCluster.InitOnCurrentMachine(4, ...)
let r =
[|for x in sourceValues -> cloud { return x |> Array.sum}|]
|> Cloud.Parallel
|> cluster.Run
|> Array.sum
cluster.KillAllWorkers()
![Page 65: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/65.jpg)
{m}braceopen MBrace.Core
...
let sourceValues = [|for x in 0 .. 16 -> [|for y in 0 .. 2000 -> rand()|]|]
let cluster = ThespianCluster.InitOnCurrentMachine(4, ...)
let r =
[|for x in sourceValues -> cloud { return x |> Array.sum}|]
|> Cloud.Parallel
|> cluster.Run
|> Array.sum
cluster.KillAllWorkers()
![Page 66: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/66.jpg)
{m}braceopen MBrace.Core
...
let sourceValues = [|for x in 0 .. 16 -> [|for y in 0 .. 2000 -> rand()|]|]
let cluster = ThespianCluster.InitOnCurrentMachine(4, ...)
let r =
[|for x in sourceValues -> cloud { return x |> Array.sum}|]
|> Cloud.Parallel
|> cluster.Run
|> Array.sum
cluster.KillAllWorkers()
![Page 67: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/67.jpg)
{m}brace
![Page 68: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/68.jpg)
{m}braceopen MBrace.Core
...
let sourceValues = [|for x in 0 .. 16 -> [|for y in 0 .. 2000 -> rand()|]|]
let cluster = ThespianCluster.InitOnCurrentMachine(4, ...)
let r =
[|for x in sourceValues -> cloud { return x |> Array.sum}|]
|> Cloud.Parallel
|> cluster.Run
|> Array.sum
cluster.KillAllWorkers()
![Page 69: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/69.jpg)
{m}braceopen MBrace.Core
...
let sourceValues = [|for x in 0 .. 16 -> [|for y in 0 .. 2000 -> rand()|]|]
let cluster = ThespianCluster.InitOnCurrentMachine(4, ...)
let r =
[|for x in sourceValues -> cloud { return x |> Array.sum}|]
|> Cloud.Parallel
|> cluster.Run
|> Array.sum
cluster.KillAllWorkers()
![Page 70: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/70.jpg)
{m}braceopen MBrace.Core
...
let sourceValues = [|for x in 0 .. 16 -> [|for y in 0 .. 2000 -> rand()|]|]
let cluster = ThespianCluster.InitOnCurrentMachine(4, ...)
let r =
sourceValues
|> CloudFlow.OfArray
|> CloudFlow.map (fun x -> x |> Array.sum )
|> CloudFlow.reduce (+)
|> cluster.Run
cluster.KillAllWorkers()
![Page 71: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/71.jpg)
{m}braceopen MBrace.Core
...
let sourceValues = [|for x in 0 .. 16 -> [|for y in 0 .. 2000 -> rand()|]|]
let cluster = ThespianCluster.InitOnCurrentMachine(4, ...)
let r =
sourceValues
|> CloudFlow.OfArray
|> CloudFlow.map (fun x -> x |> Array.sum )
|> CloudFlow.reduce (+)
|> cluster.Run
cluster.KillAllWorkers()
![Page 72: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/72.jpg)
{m}braceopen MBrace.Core
...
let sourceValues = [|for x in 0 .. 16 -> [|for y in 0 .. 2000 -> rand()|]|]
let cluster = ThespianCluster.InitOnCurrentMachine(4, ...)
let r =
sourceValues
|> CloudFlow.OfArray
|> CloudFlow.map (fun x -> x |> Array.sum )
|> CloudFlow.reduce (+)
|> cluster.Run
cluster.KillAllWorkers()
![Page 73: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/73.jpg)
{m}braceopen MBrace.Core
...
let sourceValues = [|for x in 0 .. 16 -> [|for y in 0 .. 2000 -> rand()|]|]
let cluster = ThespianCluster.InitOnCurrentMachine(4, ...)
let r =
sourceValues
|> CloudFlow.OfArray
|> CloudFlow.map (fun x -> x |> Array.sum )
|> CloudFlow.reduce (+)
|> cluster.Run
cluster.KillAllWorkers()
![Page 74: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/74.jpg)
{m}braceopen MBrace.Core
...
let sourceValues = [|for x in 0 .. 16 -> [|for y in 0 .. 2000 -> rand()|]|]
let cluster = ThespianCluster.InitOnCurrentMachine(4, ...)
let r =
sourceValues
|> CloudFlow.OfArray
|> CloudFlow.map (fun x -> x |> Array.sum )
|> CloudFlow.reduce (+)
|> cluster.Run
cluster.KillAllWorkers()
![Page 75: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/75.jpg)
3 манипулирование данными- repl & ( )- Deedle- Math.Net- R Provider- {m}brace
![Page 76: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/76.jpg)
3 манипулирование данными- repl & ( )- Deedle- Math.Net- R Provider- {m}brace- Accord.Net Framework- numl- encog- Hype- ML от MS- AForge.Net- ...
![Page 77: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/77.jpg)
4 интеграция
![Page 78: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/78.jpg)
это dotnet
![Page 79: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/79.jpg)
итог
![Page 80: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/80.jpg)
все прекрасно?
![Page 81: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/81.jpg)
нет
![Page 82: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/82.jpg)
нет, но можно
![Page 83: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/83.jpg)
f#FSharp.DataRProvider{m}brace
![Page 84: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/84.jpg)
python vs dotnet
![Page 85: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/85.jpg)
почитать- http://fsharp.org/- https://fslab.org/- http://fsharp.github.io/FSharp.Data/- http://bluemountaincapital.github.io/Deedle/- http://bluemountaincapital.github.io/FSharpRProvider/- https://numerics.mathdotnet.com/- https://tahahachana.github.io/XPlot/- https://fslab.org/FSharp.Charting/- http://accord-framework.net - http://numl.net/
![Page 86: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/86.jpg)
почитать- http://www.heatonresearch.com/encog/- https://azure.microsoft.com/ru-ru/services/machine-learning-studio/- http://hypelib.github.io/Hype/ - http://mbrace.io/
![Page 87: Артём Акуляков - F# for Data Analysis](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5a64b58b7f8b9aa6148b4627/html5/thumbnails/87.jpg)
почитать