Skip to content

stream: writable state bitmap - #49899

Merged
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
nxtedition:writable-state-bimap
Sep 29, 2023
Merged

stream: writable state bitmap#49899
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
nxtedition:writable-state-bimap

Conversation

@ronag

@ronag ronag commented Sep 27, 2023

Copy link
Copy Markdown
Member

Just some ideas. Not sure I will have time to finish this. Feel free to take over.
@ronag
ronag requested a review from benjamingr September 27, 2023 16:26
@nodejs-github-bot

nodejs-github-bot commented Sep 27, 2023

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/streams
@nodejs-github-bot nodejs-github-bot added the needs-ci PRs that need a full CI run. label Sep 27, 2023
@ronag
ronag force-pushed the writable-state-bimap branch 3 times, most recently from de32558 to 9529f21 Compare September 27, 2023 16:30
rluvaton
rluvaton reviewed Sep 27, 2023
View reviewed changes
Comment thread lib/internal/streams/writable.js Outdated
@ronag
ronag force-pushed the writable-state-bimap branch from 9529f21 to 59c24b0 Compare September 28, 2023 07:24
benjamingr
benjamingr reviewed Sep 28, 2023
View reviewed changes
Comment thread lib/internal/streams/writable.js Outdated
benjamingr
benjamingr reviewed Sep 28, 2023
View reviewed changes
Comment thread lib/internal/streams/writable.js Outdated
@benjamingr

benjamingr commented Sep 28, 2023

Copy link
Copy Markdown
Member

https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1424/
@ronag
ronag force-pushed the writable-state-bimap branch 13 times, most recently from 048fcdf to d7b1625 Compare September 28, 2023 08:14
@rluvaton

rluvaton commented Sep 28, 2023

Copy link
Copy Markdown
Member

New benchmark CI

https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1425/
@mcollina mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Sep 28, 2023
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Sep 28, 2023
@nodejs-github-bot

nodejs-github-bot commented Sep 28, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/54316/
@ronag
ronag force-pushed the writable-state-bimap branch 2 times, most recently from 38b4a3f to c39b474 Compare September 28, 2023 09:19
mcollina
mcollina approved these changes Sep 29, 2023
View reviewed changes

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

huge win. clearBuffer is one of the biggest bottlenecks in http!
@benjamingr

benjamingr commented Sep 29, 2023

Copy link
Copy Markdown
Member

Wait until you see what we have planned for optimizing state.buffered :D
@ronag

ronag commented Sep 29, 2023

Copy link
Copy Markdown
Member Author

Wait until you see what we have planned for optimizing state.buffered :D

No spoilers!
rluvaton
rluvaton reviewed Sep 29, 2023
View reviewed changes
Comment thread lib/internal/streams/writable.js
rluvaton
rluvaton reviewed Sep 29, 2023
View reviewed changes
Comment thread lib/internal/streams/writable.js
rluvaton
rluvaton approved these changes Sep 29, 2023
View reviewed changes

@rluvaton rluvaton left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well done!
@rluvaton rluvaton added stream Issues and PRs related to Node.js streams. performance Issues and PRs related to the performance of Node.js. needs-benchmark-ci PRs that need a benchmark CI run. labels Sep 29, 2023
@nodejs-github-bot

nodejs-github-bot commented Sep 29, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/54361/
@ronag

ronag commented Sep 29, 2023

Copy link
Copy Markdown
Member Author

https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1429/console
@ronag

ronag commented Sep 29, 2023

Copy link
Copy Markdown
Member Author 
17:14:01                                                                                          confidence improvement accuracy (*)   (**)   (***)
17:14:01 streams/creation.js kind='duplex' n=50000000                                                    ***     -8.37 %       ±2.83% ±3.81%  ±5.02%
17:14:01 streams/creation.js kind='readable' n=50000000                                                           0.43 %       ±0.82% ±1.09%  ±1.42%
17:14:01 streams/creation.js kind='transform' n=50000000                                                 ***      5.53 %       ±2.22% ±2.97%  ±3.91%
17:14:01 streams/creation.js kind='writable' n=50000000                                                  ***      8.52 %       ±1.75% ±2.34%  ±3.06%
17:14:01 streams/destroy.js kind='duplex' n=1000000                                                      ***     18.11 %       ±7.45% ±9.97% ±13.10%
17:14:01 streams/destroy.js kind='readable' n=1000000                                                            -0.95 %       ±2.66% ±3.54%  ±4.61%
17:14:01 streams/destroy.js kind='transform' n=1000000                                                   ***     12.84 %       ±6.31% ±8.40% ±10.93%
17:14:01 streams/destroy.js kind='writable' n=1000000                                                    ***      6.03 %       ±2.95% ±3.92%  ±5.11%
17:14:01 streams/pipe-object-mode.js n=5000000                                                                    4.20 %       ±4.66% ±6.21%  ±8.09%
17:14:01 streams/pipe.js n=5000000                                                                         *      3.85 %       ±3.16% ±4.21%  ±5.48%
17:14:01 streams/readable-async-iterator.js sync='no' n=100000                                                    0.03 %       ±3.28% ±4.37%  ±5.69%
17:14:01 streams/readable-async-iterator.js sync='yes' n=100000                                            *      6.32 %       ±5.55% ±7.42%  ±9.72%
17:14:01 streams/readable-bigread.js n=1000                                                                      -0.77 %       ±2.72% ±3.62%  ±4.72%
17:14:01 streams/readable-bigunevenread.js n=1000                                                                -5.28 %       ±5.59% ±7.44%  ±9.69%
17:14:01 streams/readable-boundaryread.js type='buffer' n=2000                                                    1.21 %       ±2.04% ±2.72%  ±3.55%
17:14:01 streams/readable-boundaryread.js type='string' n=2000                                                    0.33 %       ±0.59% ±0.79%  ±1.03%
17:14:01 streams/readable-from.js n=10000000                                                                      0.36 %       ±2.37% ±3.16%  ±4.12%
17:14:01 streams/readable-readall.js n=5000                                                                       0.73 %       ±1.85% ±2.47%  ±3.24%
17:14:01 streams/readable-unevenread.js n=1000                                                                    0.15 %       ±2.02% ±2.68%  ±3.49%
17:14:01 streams/writable-manywrites.js len=1024 callback='no' writev='no' sync='no' n=100000              *      2.17 %       ±1.88% ±2.52%  ±3.32%
17:14:01 streams/writable-manywrites.js len=1024 callback='no' writev='no' sync='yes' n=100000           ***      8.46 %       ±3.42% ±4.56%  ±5.94%
17:14:01 streams/writable-manywrites.js len=1024 callback='no' writev='yes' sync='no' n=100000                   -0.55 %       ±3.00% ±3.99%  ±5.19%
17:14:01 streams/writable-manywrites.js len=1024 callback='no' writev='yes' sync='yes' n=100000          ***      7.85 %       ±2.48% ±3.30%  ±4.30%
17:14:01 streams/writable-manywrites.js len=1024 callback='yes' writev='no' sync='no' n=100000                   -1.87 %       ±2.39% ±3.18%  ±4.14%
17:14:01 streams/writable-manywrites.js len=1024 callback='yes' writev='no' sync='yes' n=100000          ***     -5.02 %       ±2.68% ±3.57%  ±4.66%
17:14:01 streams/writable-manywrites.js len=1024 callback='yes' writev='yes' sync='no' n=100000                   0.50 %       ±2.31% ±3.07%  ±4.01%
17:14:01 streams/writable-manywrites.js len=1024 callback='yes' writev='yes' sync='yes' n=100000          **      4.24 %       ±2.61% ±3.48%  ±4.53%
17:14:01 streams/writable-manywrites.js len=32768 callback='no' writev='no' sync='no' n=100000                    1.17 %       ±3.40% ±4.53%  ±5.91%
17:14:01 streams/writable-manywrites.js len=32768 callback='no' writev='no' sync='yes' n=100000          ***     12.55 %       ±2.83% ±3.80%  ±5.01%
17:14:01 streams/writable-manywrites.js len=32768 callback='no' writev='yes' sync='no' n=100000                   0.45 %       ±1.42% ±1.89%  ±2.46%
17:14:01 streams/writable-manywrites.js len=32768 callback='no' writev='yes' sync='yes' n=100000         ***      8.27 %       ±3.76% ±5.02%  ±6.56%
17:14:01 streams/writable-manywrites.js len=32768 callback='yes' writev='no' sync='no' n=100000                  -1.06 %       ±2.82% ±3.75%  ±4.88%
17:14:01 streams/writable-manywrites.js len=32768 callback='yes' writev='no' sync='yes' n=100000          **      3.53 %       ±2.24% ±2.99%  ±3.94%
17:14:01 streams/writable-manywrites.js len=32768 callback='yes' writev='yes' sync='no' n=100000           *     -3.02 %       ±2.39% ±3.19%  ±4.18%
17:14:01 streams/writable-manywrites.js len=32768 callback='yes' writev='yes' sync='yes' n=100000                 1.87 %       ±2.48% ±3.32%  ±4.36%
@nodejs-github-bot

nodejs-github-bot commented Sep 29, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/54381/
@ronag ronag added the commit-queue PRs queued for automated landing through the Commit Queue. label Sep 29, 2023
@nodejs-github-bot nodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Sep 29, 2023
@nodejs-github-bot
nodejs-github-bot merged commit 53b5545 into nodejs:main Sep 29, 2023
@nodejs-github-bot

nodejs-github-bot commented Sep 29, 2023

Copy link
Copy Markdown
Collaborator

Landed in 53b5545
This was referenced Sep 30, 2023
Open
Open
@github-actions github-actions Bot mentioned this pull request Oct 2, 2023
Open
12 tasks
@RafaelGSS RafaelGSS mentioned this pull request Oct 9, 2023
Merged
targos pushed a commit that referenced this pull request Nov 11, 2023
@ronag @targos
stream: writable state bitmap
35ec931 
PR-URL: #49899
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
@targos targos mentioned this pull request Nov 12, 2023
Merged
@juanarbol juanarbol mentioned this pull request Dec 6, 2023
Merged
@dead-claudia dead-claudia mentioned this pull request Feb 12, 2024
Closed
@orgads orgads mentioned this pull request May 28, 2024
Closed
@orgads

orgads commented May 28, 2024

Copy link
Copy Markdown
Contributor

@ronag This causes #52228.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-benchmark-ci PRs that need a benchmark CI run. needs-ci PRs that need a full CI run. performance Issues and PRs related to the performance of Node.js. stream Issues and PRs related to Node.js streams.

7 participants

@ronag @nodejs-github-bot @benjamingr @rluvaton @orgads @mcollina @anonrig