Linq og PLinq
Plinq PresentationSteen L.
[email protected]@gmail.com?Datalog fra
rhusSpeciale: Neural NetworksDDE /
EhusetKMDBluetagsStatsbilioteketSaxotechGatehouseNordjysk
Elhandel
Den frste multi cpu computer jeg fik mulighed for at bruge var
en DDE SuperMax SVR4.2 MPmulti
2NEAS.DKNordjysk ElhandelPris aftaler el forbrug Pris aftaler el
produktion Klima Trading
Omkring 180 ansatte Handel med el el til 3AgendaQuick Linq
IntroPlinq IntroPlinq PitfallsTask Parallel Library
Quick Linq IntroSlides by Paul LitwinPLinqParrallel LinqMake it
easy for the developer to take advantage of multiple
cores.Declarative way of expressing parrallel computations
Parrallel udfrt Programmerings Sprogs integrede
foresprgelserSkal udtrykke hvad man vil ha som resultat ikke
hvordan det gresDeklarativ kontra imperativFrst vil vi se p de
muligheder plinq tilbyderOg derefter vil vi se p de
begrnsninger6Thread PoolA lot of code needed to manage the plumbing
invovled in this
Standard mde at lave en lsning som udnytter flere kerner er at
man man deler arbejdet op i opgaver Man har s en opgave k En gruppe
af trde gr s i gang med at bearbejde disse opgaverResultater
afleveres s i listeKrver tit meget kode at ud vikle dette
7Plinq thread PoolSplit the enumerable list into partitions and
let the threads run the tasks
Kilden som er en enumerable deles op i dele Trdene udfrer
beregningerneDerefter flettes resultaterne sammen
8AsParrallel.AsParrallelExtension to IEnumerablepublic static
ParallelQuery AsParallel( this IEnumerable source )The Ienumerable
source should represent the tasks that should run in parrallel
9ParallelQuery.WithDegreeOfParallelism()The number of threads to
be usedExample:
PartitionerThe Plinq runtime choose how to partition the source
The default for List source is the range partitioner
Range Partitioning This is a pretty common partitioning scheme,
similar to the one that I described in the example above. This is
amenable to many query shapes, though it only works with indexible
data sources such as lists and arrays (i.e. IList and T[]). If you
give PLINQ something typed as IEnumerable or IEnumerable, PLINQ
will query for the ILIst interface, and if its found, will use that
interface implementation with range partitioning. The benefits of
these data sources is that we know the exact length and can access
any of the elements within the arrays directly. For the majority of
cases, there are large performance benefits to using this type of
partitioning.11PartitionerChunk Partitioning
Striped Partitioning
Hash Partitioning
Chunk Partitioning This is a general purpose partitioning scheme
that works for any data source, and is the main partitioning type
for non-indexible data sources. In this scheme, worker threads
request data, and it is served up to the thread in chunks.
IEnumerables and IEnumerables do not have fixed Count properties
(there is a LINQ extension method for this, but that is not the
same), so theres no way to know when or if the data source will
enumerate completely. It could be 3 elements, it could be 3 million
elements, it could be infinite. A single system needs to take all
of these possibilities into account and factor in different
delegate sizes, uneven delegate sizes, selectivity etc. The chunk
partitioning algorithm is quite general and PLINQs algorithm had to
be tuned for good performance on a wide range of queries. Another
important optimization is that chunk partitioning balances the load
among cores, as the tasks per core dynamically request more work as
needed. This ensures that all cores are utilized throughout the
query and can all cross the finish line at the same time vs. a
ragged, sequential entry to the end.12PartitionerYou can write a
custom Partitioner that inherits from PartitionerThis can tune the
performance if the partitioning is a bottleneckExample
ParallelQueryWithMergeOptions()FullyBuffered each thread process
the part, then merge
ExampleParallelQueryWithCancellation(CancellationToken)Makes it
possible to cancel the parrallel queryAsOrdered()Wait for all
threads to finish and then orders the result as a normal query
ParallelQueryForAll()Do some work for all elements but returns
nothing
ExceptionsAggregateExceptionCollects all the exceptions thrown
by the threads InnerExceptions member returns the list of
exceptions
When to use PlinqN elements - M threads - T time to process
Overhead Split the enumerable into parts O(N) Start the threads
O(M)Merge the part results into the complete result O(N)Gain
O(N/M*T)O(N/M*T) > (2*O(N)+ O(M))
Plinq and db sourcesBeware if the source for the AsParrallel is
a database source.ExampleDB use chunk partioning ToList() use range
partitioningPLINQ works with LINQ-for-Objects and LINQ-for-XML.
PLINQ isn't intended for use with LINQ-to-SQL or LINQ-to-Entities.
Why?Because the IQueryProvider implementation for SQL Server
basically translates LINQ queries to SQL queries for LINQ-to-SQL
andLINQ-to-Entities--queries that are processed by the SQL engine
instead of in memory.19Task Parallel Library
Task Parallel LibraryParallel.InvokeParallel.Invoke( () =>
MethodA(), () => MethodB(), () =>
MethodC());Parallel.ForEachParallel.ForEach(list, e => {
DoWork(e) });Parallel.ForParallel.For(start, end, (i) =>
DoSomeWork(i));
Task Parallel LibraryTaskFuture a proxy for an object not yet
computed. Task task = Task.Factory.StartNew(() =>
DoAction());
Is the task done:If(task.IsCompleted)
Wait to the job to be donetask.Wait();
In computer science, future, promise, and delay refer to
constructs used for synchronization in some concurrent programming
languages. They describe an object that acts as a proxy for a
result that is initially not known, usually because the computation
of its value has not yet completed.22
