Using the pipes library, I want to write a program to read data from some source and accumulate it monoidally (say, with Sum). The easiest way to do this would be,
import Control.Proxy as
import Data.Monoid (Sum)
main = do
let source = enumFromToS (0::Int) 5
a <- runWriterT $ runProxy $ source >-> foldD Sum
print a
Of course, while this works for small sources, large inputs will result in the dreaded stack overflow due to the lazy nature of WriterT's accumulator.
Thankfully, it seems that pipes anticipates this, providing a WriterP proxy with a strict accumulator. Unfortunately, the documentation surrounding this proxy is pretty sparse. After a bit of poking around (and simplifying the problem to instead accumulate a 1 for each downstream element), I came to this program,
import Control.Proxy
import Control.Proxy.Trans.Writer
import Data.Monoid (Sum)
main = do
let source = enumFromToS (0::Int) 5
a <- runProxy $ runWriterK $ source >-> \x->tell (Sum 1::Sum Int)
print a
Of course, this program doesn't even perform the simplified task correctly as it accumulates to 1 instead of 6. If I'm not mistaken, this behavior is explained by the fact that the pipe only reads one element before terminating. To repeat until the end of the input, I came up with the following,
import Control.Proxy
import Control.Proxy.Trans.Writer
import Data.Monoid (Sum)
main = do
let source = enumFromToS (0::Int) 5
a <- runProxy $ runWriterK $ source >-> fold Sum
print a
fold :: (Monad m, Proxy p, Monoid w) => (a -> w) -> a' -> WriterP w p a' a a' a m r
fold f = go
where go x = do a <- request x
tell $ f a
x' <- respond a
go x'
This code, however, returns an accumulator of 0. Why is this? Is there a function like my fold provided in pipes?
Given that many use-cases for pipes are long-running processes working with large data-sets, would it not make sense for the folds in Control.Proxy.Prelude to be built around the strict WriterP instead of WriterT? Currently it feels like the proxy transformers in pipes are second-class citizens, present but lacking many of the combinators that make WriterT so handy.
foldDandsumDfrom the folds coming withpipes. Looking at the source code should clarify your doubt. – Danny Navarro Jan 24 '13 at 18:39StateT, I assume. – C. A. McCann Jan 24 '13 at 21:47Control.Monad.Trans.Writer.Strictis something that has tripped me up numerous times in the past. I was under the impression that it wasn't even possible to write a strictly accumulating (e.g. forcing themappend)Writermonad as you have nothing on which toseq. – bgamari Jan 25 '13 at 0:03