logoalt Hacker News

win311fwg • yesterday at 4:40 PM • 1 reply • view on HN

I don't see the problem.

    for _, filename := range files {
        queue <- func() {
            f, _ := os.Open(filename)
            defer f.Close()
        }
    }
or more realistically,

    var group errgroup.Group
    group.SetLimit(10)
    for _, filename := range files {
        group.Go(func() error {
            f, err := os.Open(filename)
            if err != nil {
                return fmt.Errorf("failed to open file %s: %w", filename, err)
            }
            defer f.Close()  
            // ...
            return nil          
        })
    }
    if err := group.Wait(); err != nil {
        return fmt.Errorf("failed to process files: %w", err)
    }
Perhaps you can elaborate?

I did read your code, but it is not clear where the worker queue is. It looks like it ranges over (presumably) a channel of filenames, which is not meaningfully different than ranging over a slice of filenames. That is the original, non-concurrent solution, more or less.

Replies

mort96 • yesterday at 5:24 PM

I think they imagine a solution like this:

    // Spawn workers
    for _ := range 10 {
        go func() {
            for path := range workQueue {
                fp, err := os.Open(path)
                if err != nil { ... }
                defer fp.Close()
                // do work
            }
        }()
    }

    // Iterate files and give work to workers
    for _, path := range paths {
        workQueue <- path
    }
➕ show 1 reply