Skip to content

Adds sub-command clear-session to history command. Issue #5791#8337

Closed
s7rul wants to merge 6 commits into
fish-shell:masterfrom
s7rul:history-clear-session-feature
Closed

Adds sub-command clear-session to history command. Issue #5791#8337
s7rul wants to merge 6 commits into
fish-shell:masterfrom
s7rul:history-clear-session-feature

Conversation

@s7rul
Copy link
Copy Markdown
Contributor

@s7rul s7rul commented Oct 7, 2021

Description

Adds sub-command clear-session to history command. Which clear current session history. If history merge or builtin history merge is run in a session only the history after this will be erased.

Implementation

A function clear_session witch does this is added to src/history.cpp where it first puts all of the commands in new_items into deleted_itemsafter this new_items is cleared and first_unwritten_new_item_index is reset.

Documentation is updated in doc_src and completion. Tests were written for the added code (no unit test in fish_tests.cpp).

Fixes issue #5791

Additional notes

We have done this as part in a school project and it is our first contribution to open-source.

TODO

  • Make test pass
  • Make it not affect old history via de-duplication.



This clears all history for the current session. Documentation and autocompletion for the new subcommand is added.
@s7rul s7rul force-pushed the history-clear-session-feature branch from 5a3c6ac to 53bb36a Compare October 7, 2021 14:58
@faho
Copy link
Copy Markdown
Member

faho commented Oct 7, 2021

We have done this as part in a school project and it is our first contribution to open-source.

Oooh that's super cool!

We probably want @ridiculousfish to review this as there are some subtleties to the history code.

I'm not quite sure why the test is failing, you can see how, in the failing tests, the history prints nothing:

Escaped buffer:
\rprompt 34>history\r\n⏎                                                                              \r⏎ \r\rprompt 35>

This can be worked around by adding a sleep(1) in line 164 of the test, after the expect_prompt() for the history clear.

I think what happens is that the clear takes effect later, after the other commands have been run. Free CI systems are often quite underpowered and overloaded, especially things like Github's macOS runners, so you'll often see failures like this there that you'd never see on your local machine or in actual use.

Since this implies some sort of race condition in the reader/history interactions (or possibly it's intentional that these changes are only loaded after a while to avoid burning CPU?), I don't believe it is something for you to solve, so try to just stick the sleep in there for now.


I have one quibble with the way this works:

Fish deduplicates history. That has major implications for this, because, as it currently seems to work, if I do

> ls
> history clear-session

That clear-session will erase all traces of an exact command ls from the entire history. That doesn't seem like a good idea as it would also delete things like cd Videos (which I constantly do!), which would make autosuggestions quite a bit less useful. I think it would be better if ls would only be deleted if it was new in that session. Unfortunately I don't believe just not adding to deleted_items would be enough. You might have to either keep a separate list of the actually new items (which would require memory) or possibly check for duplicates when clear-session is done, and keep any duplicates around.

@s7rul
Copy link
Copy Markdown
Contributor Author

s7rul commented Oct 7, 2021

I'm not quite sure why the test is failing, you can see how, in the failing tests, the history prints nothing:

Escaped buffer:
\rprompt 34>history\r\n⏎                                                                              \r⏎ \r\rprompt 35>

This can be worked around by adding a sleep(1) in line 164 of the test, after the expect_prompt() for the history clear.

I think what happens is that the clear takes effect later, after the other commands have been run. Free CI systems are often quite underpowered and overloaded, especially things like Github's macOS runners, so you'll often see failures like this there that you'd never see on your local machine or in actual use.

Since this implies some sort of race condition in the reader/history interactions (or possibly it's intentional that these changes are only loaded after a while to avoid burning CPU?), I don't believe it is something for you to solve, so try to just stick the sleep in there for now.

I managed to make it pass just before i saw your response. The sleep seems like a better solution though.

I have one quibble with the way this works:

Fish deduplicates history. That has major implications for this, because, as it currently seems to work, if I do

> ls
> history clear-session

That clear-session will erase all traces of an exact command ls from the entire history. That doesn't seem like a good idea as it would also delete things like cd Videos (which I constantly do!), which would make autosuggestions quite a bit less useful. I think it would be better if ls would only be deleted if it was new in that session. Unfortunately I don't believe just not adding to deleted_items would be enough. You might have to either keep a separate list of the actually new items (which would require memory) or possibly check for duplicates when clear-session is done, and keep any duplicates around.

Did not think of that but agree that it would be better if it did not touche old history in any way.

@ridiculousfish ridiculousfish self-requested a review October 8, 2021 21:35
@ridiculousfish
Copy link
Copy Markdown
Member

This is great! Impressively simple implementation, nice tests too. I am able to reproduce the test failures locally, I will investigate to ensure we aren't masking some serious bug.

Regarding touching old history, faho is correct that this will erase items from previous sessions, if the item is also in the current session. I think this can be fixed easily by tracking in deleted_items whether the deletion is for all sessions or just this session, and then inspect that field when de-duplicating to decide whether to erase. If you'd like to tackle that please do; otherwise I will have a go and add on top of this commit.

@zx8
Copy link
Copy Markdown

zx8 commented Oct 9, 2021

Also perhaps history clear --session instead?

@s7rul
Copy link
Copy Markdown
Contributor Author

s7rul commented Oct 9, 2021

Regarding touching old history, faho is correct that this will erase items from previous sessions, if the item is also in the current session. I think this can be fixed easily by tracking in deleted_items whether the deletion is for all sessions or just this session, and then inspect that field when de-duplicating to decide whether to erase. If you'd like to tackle that please do; otherwise I will have a go and add on top of this commit.

I can give it a try, sounds simple enough.

@ridiculousfish
Copy link
Copy Markdown
Member

ridiculousfish commented Oct 10, 2021

So the test failures were due to this hack and not a problem with this PR.

The gory details: fish shares a single history file with all other instances. Any instance should only "see" past items, not items from other running instances. It does this via timestamps: a fish instance has a start timestamp. Items with earlier timestamps are "old" and should be considered part of history. Items with later timestamps are "new" and should be ignored by this instance.

A problem arises with items whose timestamp exactly matches fish's start date: those items may be old or new. For example exec fish is likely to create an "old" history item that exactly matches fish's start date. Meanwhile scripted tests also create history items in the same second as fish launches. To distinguish these, any new history item created in the same second as fish starts will have its timestamp incremented by one.

The original tests failed because the item gets written with a timestamp in the future. When fish re-reads the item it then thinks it's from another concurrently running fish instance. This also explains why adding a sleep 1 at the beginning of the tests fixes them: it pushes the history item's timestamp past fish's birth timestamp.

Anyways we're dealing with the consequences of my gross hack. I'll see if I can fix this another way but it shouldn't block merge.

@ridiculousfish
Copy link
Copy Markdown
Member

Also perhaps history clear --session instead?

I prefer a separate command clear-session. Usually you want commands to do the least dangerous thing and for modifiers to make them more dangerous, e.g. rm vs rm -R. But with history clear --session it would be reversed: the default would be more destructive. So I think a separate command is better in this case.

Copy link
Copy Markdown
Member

@ridiculousfish ridiculousfish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM. Let us know should you decide to hand off the "don't delete old history" changes.

Comment thread src/history.cpp Outdated
@floam floam added this to the fish 3.4.0 milestone Oct 11, 2021
set hist_cmd search
else if set -q _flag_merge
set hist_cmd merge
else if set -q _flag_clear-session
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
else if set -q _flag_clear-session
else if set -q _flag_clear_session

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know how to think about this. On one hand my version breaks snake case on the other hand with full snake case the literal flag name is not there.

Fixed problem where if an old command where the same as a new command clear_session would erase them both.
This will now only occur with commands from a different fish session typed after the first session is started.
Comment thread src/history.cpp Outdated
@s7rul
Copy link
Copy Markdown
Contributor Author

s7rul commented Oct 13, 2021

I have now made a partial solution to this problem:

That clear-session will erase all traces of an exact command ls from the entire history. That doesn't seem like a good idea as it would also delete things like cd Videos (which I constantly do!), which would make autosuggestions quite a bit less useful. I think it would be better if ls would only be deleted if it was new in that session. Unfortunately I don't believe just not adding to deleted_items would be enough. You might have to either keep a separate list of the actually new items (which would require memory) or possibly check for duplicates when clear-session is done, and keep any duplicates around.

The problem with my solution other then that it is a little bit messy in my opinion is that it only solves the problem for commands with a timestamp older then the current session. I don't know to solve this without changing substantial parts of the history subsystem and/or adding a session identifier to the commands in the history file. If anyone have a solution or suggestion it is welcome.

@ridiculousfish
Copy link
Copy Markdown
Member

The problem with my solution other then that it is a little bit messy in my opinion is that it only solves the problem for commands with a timestamp older then the current session.

I think that's fine - to only preserve items that were already present. To be precise, the problem is:

  1. Launch fish in tty1
  2. Launch fish in tty2
  3. Run the same command in both
  4. history clear-session in tty1
  5. The item is erased from history even though it was also run in tty2

Some day we will fix the history format, but for now we can totally live with this.

@s7rul
Copy link
Copy Markdown
Contributor Author

s7rul commented Oct 14, 2021

The problem with my solution other then that it is a little bit messy in my opinion is that it only solves the problem for commands with a timestamp older then the current session.

I think that's fine - to only preserve items that were already present. To be precise, the problem is:

1. Launch fish in tty1

2. Launch fish in tty2

3. Run the same command in both

4. `history clear-session` in tty1

5. The item is erased from history even though it was also run in tty2

Some day we will fix the history format, but for now we can totally live with this.

Then i consider this to be solved.

@floam
Copy link
Copy Markdown
Member

floam commented Oct 17, 2021

adding a session identifier to the commands in the history file

Long term I think doing this is not a bad idea. It will enable making fish capable of "resuming" a specified session after the fact.

Even Terminal.app exports e.g. TERM_SESSION_ID=6B538BF1-6106-40D8-A9C0-71E0D6D6B21A and bash/zsh are able to cause the history to match the saved scrollback for resumed tabs/windows. I think iTerm has some mechanism for this as well.

@ridiculousfish
Copy link
Copy Markdown
Member

Looks good, squash-merged as 049104e. Thank you, a wonderful contribution!

@s7rul s7rul deleted the history-clear-session-feature branch October 18, 2021 17:28
krobelus added a commit to krobelus/fish-shell that referenced this pull request Dec 12, 2021
I used the command from fish-shell#8092 to list issues/PRs with missing changelog entries,
and went through most of them.

Each issue gets three lines
- subject
- URL
- verdict

If the verdict ends with ", ignoring", I added it the the ignored list in the changelog.

The issues are grouped by verdict, with the interesting/leftover ones
on top (obviously no need to double check *everything*).

The "gh" script is already a quantum leap we can still find better
ways to share the changelog burden.
I noticed that there are many minor updates that can probably be
ignored.  Filtering them out doesn't take much time but it adds up,
especially if it's a single person doing it.

---

Issue/PR title: Make --no-config mode more comfortable
fish-shell#8493
in-flight

Issue/PR title: Debian fish binary package difficult to install
fish-shell#7845
TODO

Issue/PR title: Work around `setpgid` error on older Apple platforms
fish-shell#8153
this extends 7474, which wasn't listed either, we should probably mention it?

Issue/PR title: Abbr -q return status inconsistent
fish-shell#8431
fixes return status of "abbr -q", should we mention this?

Issue/PR title: math: (n n): incorrect error
fish-shell#8511
improved error output, which is very nice but too minor?

Issue/PR title: Fish autocomplete error on iOS procursus
fish-shell#8205
niche fix, ignoring

Issue/PR title: Fix `fish_key_reader` wrapper check
fish-shell#8271
minor update to not create a harmless alias for fish_key_reader, ignoring

Issue/PR title: funced dosn't like backslash escapes in function names
fish-shell#8289
minor escaping fix, ignoring

Issue/PR title: Hide whatis database building from the user
fish-shell#8310
not something many users would notice in the first place, ignoring

Issue/PR title: Duplicated "Type 'help argparse' for related documentation" for argparse
fish-shell#8368
minor update to error message, ignoring

Issue/PR title: Variable highlight color does not span lines
fish-shell#8444
very obscure fix, ignoring

---

Issue/PR title: Add --function to `read`
fish-shell#8295
added to existing entry (565)

Issue/PR title: Added completions for ethtool
fish-shell#8283
added to existing entry

Issue/PR title: Add dart completion
fish-shell#8315
added to existing entry

Issue/PR title: Add common lisp completions(sbcl/roswell)
fish-shell#8330
added to existing entry

Issue/PR title: Fix st issue with shift+tab
fish-shell#8354
added to existing entry (8352)

Issue/PR title: Support vi-mode cursors in Foot Terminal 
fish-shell#8391
added to existing entry (8167)

Issue/PR title: Completions pager should redraw if the subbed completion wraps/unwraps the line
fish-shell#8405
added to existing entry (8509)

---

Issue/PR title: Binding escape as user binding breaks escape sequence bindings (arrows, etc)
fish-shell#8428
added new entry

Issue/PR title: Windows "color" command completion
fish-shell#8483
added new entry

Issue/PR title: Doesn't build when using netbsd curses on Linux
fish-shell#8087
added new entry

Issue/PR title: Don't override linker
fish-shell#8152
added new entry

Issue/PR title: Add completions for `git-sizer`
fish-shell#8156
added new entry

Issue/PR title: `d3ceba107e88b6c6e1a0358ebcb30366aeef653f` causes issues with repainting multi-line prompt
fish-shell#8163
added new entry

Issue/PR title: Builtin math ncr can be extremely slow
fish-shell#8170
added new entry

Issue/PR title: Completion sometimes missing the last token
fish-shell#8175
added new entry

Issue/PR title: `set -S` should mark read-only variables
fish-shell#8179
added new entry

Issue/PR title: Errors when trying to autocomplete (invalid) UTF-8 escapes
fish-shell#8195
added new entry

Issue/PR title: Always use LC_NUMERIC=C internally
fish-shell#8204
added new entry

Issue/PR title: Slow interaction between backgrounding, universal variables, and repainting
fish-shell#8209
added new entry

Issue/PR title: Unsetting `$fish_emoji_width` doesn't clear the cached width
fish-shell#8274
added new entry

Issue/PR title: If prompt ends in an empty line, the commandline is inserted at the width of the line before
fish-shell#8298
added new entry

Issue/PR title: assertion normal_exited() failed related to paged builtin help
fish-shell#8308
added new entry

Issue/PR title: colors don't kick in for ls on macOS Big Sur, Monterey (and maybe FreeBSD)
fish-shell#8309
added new entry

Issue/PR title: Adds sub-command clear-session to history command. Issue fish-shell#5791
fish-shell#8337
added new entry (as 5791)

Issue/PR title: Display local branches before unique remote branches in git completion
fish-shell#8338
added new entry

Issue/PR title: Fix delete-key in st
fish-shell#8352
added new entry

Issue/PR title: sigsegv on set --show variable (when LANG is set to fr_FR.utf8)
fish-shell#8358
added new entry

Issue/PR title: Add clasp completion
fish-shell#8373
added new entry

Issue/PR title: `cargo run --example` completions break with nested example directories
fish-shell#8429
added new entry

Issue/PR title: argparse completions
fish-shell#8434
added new entry

Issue/PR title: Stop linking to StackOverflow
fish-shell#8495
added new entry

Issue/PR title: fish_key_reader ^C warning isn't right
fish-shell#8510
added new entry

Issue/PR title: Use `--almost-all` in `la` function
fish-shell#8519
added new entry

---

Issue/PR title: improve the experience of using fish over mosh
fish-shell#1363
listed as 8376, ignoring

Issue/PR title: incomplete man page completions
fish-shell#8305
listed as 8309, ignoring

Issue/PR title: Support "$(cmd)" command substitution without line splitting
fish-shell#8059
listed as 159, ignoring

Issue/PR title: fish_config: Read colorschemes from .theme files
fish-shell#8127
listed as 8132, ignoring

Issue/PR title: funced: edit the whole file, not just the function definition
fish-shell#8130
listed as 391, ignoring

Issue/PR title: builtin cd: print error about broken symlink
fish-shell#8270
listed as 8264, ignoring

Issue/PR title: fix man completion for BSD's mandoc
fish-shell#8306
listed as 8305, ignoring

Issue/PR title: Don't escape tildes that come from custom completions
fish-shell#8441
listed as 8441, ignoring

Issue/PR title: Use `cargo run --example` to get list of examples
fish-shell#8446
listed as 8429, ignoring

---

Issue/PR title: Node completion: add v8 sparkplug option
fish-shell#8118
update to existing completions, ignoring

Issue/PR title: Add zypper subcommands completion
fish-shell#8183
update to existing completions, ignoring

Issue/PR title: completion nmap: suppress warning when local scripts folder exists
fish-shell#8184
update to existing completions, ignoring

Issue/PR title: add missing `git commit` completions
fish-shell#8191
update to existing completions, ignoring

Issue/PR title: Updated ping completions
fish-shell#8192
update to existing completions, ignoring

Issue/PR title: Add `--function` to `set` completion
fish-shell#8202
update to existing completions, ignoring

Issue/PR title: completion: support `--no` prefixes for mpv flag options
fish-shell#8219
update to existing completions, ignoring

Issue/PR title: complete "mpc load"
fish-shell#8241
update to existing completions, ignoring

Issue/PR title: Add and fix completions for new options
fish-shell#8243
update to existing completions, ignoring

Issue/PR title: Fix completions/ls.fish
fish-shell#8249
update to existing completions, ignoring

Issue/PR title: Fix completions/coredumpctl.fish and add new complete
fish-shell#8256
update to existing completions, ignoring

Issue/PR title: completions/git: Handle "1 .T" & "1 AT" files
fish-shell#8311
update to existing completions, ignoring

Issue/PR title: completions/xbps-query: add missing `-p` completions
fish-shell#8323
update to existing completions, ignoring

Issue/PR title: Update ldapsearch.fish
fish-shell#8326
update to existing completions, ignoring

Issue/PR title: small fix completions/duply.fish
fish-shell#8327
update to existing completions, ignoring

Issue/PR title: Update ip.fish
fish-shell#8334
update to existing completions, ignoring

Issue/PR title: Fix ant completion
fish-shell#8344
update to existing completions, ignoring

Issue/PR title: Update dmesg completions
fish-shell#8365
update to existing completions, ignoring

Issue/PR title: No hints for -g|--global and -U|--universal flags for abbr command
fish-shell#8367
update to existing completions, ignoring

Issue/PR title: Updated systemd-analyze completions
fish-shell#8381
update to existing completions, ignoring

Issue/PR title: vmctl completion function call needs to be quoted
fish-shell#8406
update to existing completions, ignoring

Issue/PR title: pabcnetcclear command completion update
fish-shell#8480
update to existing completions, ignoring

---

Issue/PR title: document `--no-config`
fish-shell#8176
doc update, ignoring

Issue/PR title: Theme demo needs to be adjusted so that only unmatched quote is an error
fish-shell#8260
doc update, ignoring

Issue/PR title: no error about wrong >>? redirection operator
fish-shell#8380
doc update, ignoring

Issue/PR title: set -l works outside of command block
fish-shell#8385
doc update, ignoring

Issue/PR title: Some enhancements to "for" and "while" loop pages
fish-shell#8409
doc update, ignoring

Issue/PR title: Html docs: Remove link underlines again?
fish-shell#8439
doc update, ignoring

Issue/PR title: Old-style options support "=" assignment operator in complete builtin
fish-shell#8457
doc update, ignoring

Issue/PR title: Document prompt_hostname
fish-shell#8522
doc update, ignoring

---

Issue/PR title: edit_command_buffer: use "command" to ignore any functions with the same name
fish-shell#8221
only helps broken systems, ignoring

Issue/PR title: Prepend command to cat
fish-shell#8287
only helps broken systems, ignoring

Issue/PR title: Make less version check compatible with older Fish
fish-shell#8299
only helps broken systems, ignoring

Issue/PR title: Skip leading `command` in `__fish_man_page`
fish-shell#8447
only helps broken systems, ignoring

Issue/PR title: fish_config doesn't work without curses module
fish-shell#8487
only helps broken systems, ignoring

---

Issue/PR title: fix 'socket file name too long' error
fish-shell#8128
test fix with long tempdirs (macOS), not really user-visible, ignoring

Issue/PR title: Give tests a more generic name
fish-shell#8449
not user-visible, ignoring

Issue/PR title: string tests sometimes failing on macOS (Github Actions)
fish-shell#8353
not user-visible, ignoring

Issue/PR title: history merge test fails on OpenBSD
fish-shell#6477
not user-visible, ignoring

---

Issue/PR title: Obtain Deno completions from itself
fish-shell#8471
update to an unreleased feature (7138), ignoring

Issue/PR title: `string length --visible` performance
fish-shell#8253
update to an unreleased feature, ignoring

Issue/PR title: Backspace character is ignored when calculating string widths
fish-shell#8277
update to an unreleased feature, ignoring

Issue/PR title: `fish_config choose` leaves previous right prompt in place
fish-shell#8314
update to an unreleased feature, ignoring

Issue/PR title: parenthesis characters outer of $(command substitution) in string cause error
fish-shell#8394
update to an unreleased feature, ignoring

Issue/PR title: Parser bug with command substitutions in strings inside parenthesis
fish-shell#8500
update to an unreleased feature, ignoring

Issue/PR title: fish_config: silently doesn't set color schemes.
fish-shell#8419
regression, not in any release, ignoring

Issue/PR title: :program: in sphinx doesn't link
fish-shell#8438
regression, not in any release, ignoring

Issue/PR title: __fish_seen_argument.fish throws exception when autocompleting
fish-shell#8478
regression, not in any release, ignoring

---

Issue/PR title: Fix typo in abbr docs
fish-shell#8280
typofix, ignoring

Issue/PR title: Fix typo in `set_colors` command documentation
fish-shell#8321
typofix, ignoring

Issue/PR title: Typo funcions -> functions
fish-shell#8257
typofix, ignoring

---

Issue/PR title: remove make_pair
fish-shell#8206
no behavior change, ignoring

Issue/PR title: replace push_back with emplate_back
fish-shell#8222
no behavior change, ignoring

Issue/PR title: clang-tidy: remove pointless virtual
fish-shell#8224
no behavior change, ignoring

Issue/PR title: change value to rvalue reference
fish-shell#8227
no behavior change, ignoring

Issue/PR title: convert const ref to rvalue ref
fish-shell#8228
no behavior change, ignoring

Issue/PR title: clang-tidy: use for range loops
fish-shell#8229
no behavior change, ignoring

Issue/PR title: fix deleted constructors
fish-shell#8230
nno behavior change, ignoring

Issue/PR title: clang-tidy: const reference conversions
fish-shell#8231
no behavior change, ignoring

Issue/PR title: clang-tidy: simplify two bool returns
fish-shell#8235
no behavior change, ignoring

Issue/PR title: clang-tidy: replace size comparisons with empty
fish-shell#8237
no behavior change, ignoring

Issue/PR title: clang-tidy: replace NULL with nullptr
fish-shell#8239
no behavior change, ignoring

Issue/PR title: add constexpr
fish-shell#8252
no behavior change, ignoring

Issue/PR title: __fish_seen_subcommand_from and __fish_seen_argument update
fish-shell#8430
no behavior change (apart from a regression that's fixed), ignoring

Issue/PR title: Run fish_indent on all non-test .fish files
fish-shell#8476
no behavior change, ignoring

Issue/PR title: Use test command instead of bracket command
fish-shell#8477
no behavior change, ignoring

Issue/PR title: Fix code scanning alert - Wrong type of arguments to formatting function
fish-shell#8521
no behavior change, ignoring

Issue/PR title: clang-tidy: replace push_back with emplace_back
fish-shell#8236
no behavior change, ignoring

---
@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Oct 24, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants