client

Documentation for using streamListDiff in the browser.

IMPORT

import { streamListDiff } from "@donedeal0/superdiff/client";

streamListDiff requires ESM support for browser usage. It will work out of the box if you use a modern bundler (Webpack, Rollup) or JavaScript framework (Next.js, Vue.js).

---

FORMAT

Input

In a browser environment, ReadableStream refers to the browser's streaming API, and File refers to an uploaded or local file. Examples are provided in the #usage section below.

 prevList: ReadableStream<Record<string, unknown>> | File | Record<string, unknown>[],
 nextList: ReadableStream<Record<string, unknown>> | File | Record<string, unknown>[],
 referenceKey: keyof Record<string, unknown>,
 options: {
  showOnly?: ("added" | "deleted" | "moved" | "updated" | "equal")[], // [] by default
  chunksSize?: number, // 0 by default
  considerMoveAsUpdate?: boolean; // false by default
  useWorker?: boolean; // true by default
  showWarnings?: boolean; // true by default

}

• prevList: the original object list.

• nextList: the new object list.

• referenceKey: a key common to all objects in your lists (e.g. id).

• options

  • chunksSize the number of object diffs returned by each streamed chunk. (e.g. 0 = 1 object diff per chunk, 10 = 10 object diffs per chunk).

  • showOnly gives you the option to return only the values whose status you are interested in (e.g. ["added", "equal"]).

  • considerMoveAsUpdate: if set to true a moved value will be considered as updated.

  • useWorker: if set to true, the diff will be run in a worker for maximum performance. Only recommended for large lists (e.g. +100,000 items).

  • showWarnings: if set to true, potential warnings will be displayed in the console.

Using Readable streams may impact workers' performance since they need to be converted to arrays. Consider using arrays or files for optimal performance. Alternatively, you can turn the useWorker option off.

Output

The objects diff are grouped into arrays - called chunks - and are consumed thanks to an event listener. You have access to 3 events:

• data: to be notified when a new chunk of object diffs is available.

• finish: to be notified when the stream is finished.

• error: to be notified if an error occurs during the stream.

interface StreamListener<T> {
  on(event: "data", listener: (chunk: StreamListDiff<T>[]) => void);
  on(event: "finish", listener: () => void);
  on(event: "error", listener: (error: Error) => void);
}

type StreamListDiff<T extends Record<string, unknown>> = {
  value: T | null;
  index: number | null;
  previousValue: T | null;
  previousIndex: number | null;
  status: "added" | "deleted" | "moved" | "updated" | "equal";
};

---

USAGE

Input

Array

const diff = streamListDiff(
      [ 
        { id: 1, name: "Item 1" },  
        { id: 2, name: "Item 2" },
        { id: 3, name: "Item 3" } 
      ],
      [
        { id: 0, name: "Item 0" }, 
        { id: 2, name: "Item 2" },
        { id: 3, name: "Item Three" },
      ],
      "id", 
      { chunksSize: 2 }
    );

Stream

const previousStream = new ReadableStream({
      start(controller) {
        previousArray.forEach((value) => controller.enqueue(value));
        controller.close();
      },
    }); 
    
const nextStream = new ReadableStream({
      start(controller) {
        nextArray.forEach((value) => controller.enqueue(value));
        controller.close();
      },
    }); 
    
const diff = streamListDiff(previousStream, nextStream, "id", { chunksSize: 2 });

File

const previousFile = new File([JSON.stringify(previousArrayFile)], 
                              "previousArray.json", 
                              { type: "application/json" }); 
                              
const nextFile = new File([JSON.stringify(nextArrayFile)], 
                              "nextArray.json", 
                              { type: "application/json" }); 
                              
const diff = streamListDiff(previousFile, nextFile, "id", { chunksSize: 10 });
 

Example

const diff = streamListDiff(
      [ 
-       { id: 1, name: "Item 1" },  
        { id: 2, name: "Item 2" },
        { id: 3, name: "Item 3" } 
      ],
      [
+       { id: 0, name: "Item 0" }, 
        { id: 2, name: "Item 2" },
+       { id: 3, name: "Item Three" },
      ],
      "id", 
      { chunksSize: 2 }
    );

Output

diff.on("data", (chunk) => {
      // first chunk received (2 object diffs)
      [
+       {
+         value: { id: 0, name: "Item 0" },
+         index: 0,
+         previousValue: null,
+         previousIndex: null,
+         status: "added"
+       },
-       {
-         value: null,
-         index: null,
-         previousValue: { id: 1, name: "Item 1" },
-         previousIndex: 0,
-         status: "deleted"
-       }
      ]
    // second chunk received (2 object diffs)
      [
        {
          value: { id: 2, name: "Item 2" },
          index: 1,
          previousValue: { id: 2, name: "Item 2" },
          previousIndex: 1,
          status: "equal"
        },
+       {
+         value: { id: 3, name: "Item Three" },
+         index: 2,
+         previousValue: { id: 3, name: "Item 3" },
+         previousIndex: 2,
+         status: "updated"
+       },
     ]
});

diff.on("finish", () => console.log("Your data has been processed. The full diff is available."))
diff.on("error", (err) => console.log(err))

---

REACT

Here is an example of its use in a React application.

export function useListDiff() {
  const [list, setList] = useState<StreamListDiff<Data>[]>([]);
  const [isStreaming, setIsStreaming] = useState(false);
  const [isError, setIsError] = useState<Error>();

  const getListDiff = useCallback(async () => {
    setIsStreaming(true);
    const [prevList, nextList] = await fetchLists()
    const diff = streamListDiff(prevList, nextList, "id", {
      chunksSize: 50000, // Large chunks size for fewer updates
      useWorker: true,
    });

    diff.on("data", (chunk) => {
      // Schedule state updates on the next frame for smooth rendering
      requestAnimationFrame(() => {
        setList((prev) => [...prev, ...chunk]);
      });
    });
    diff.on("finish", () => setIsStreaming(false));
    diff.on("error", (err) => {
      setIsStreaming(false);
      setIsError(err);
    });
  }, []);

  useEffect(() => {
    getListDiff();
  }, [getListDiff]);

  return {
    list,
    isStreaming,
    isError,
  };
}