allow `git-bug` offline editing for issues

[mellowagain]

requires implementing CRDT that git-bug already does and extending our gitarena-issues crate.

things to consider:

• how do we map unknown identities to our web frontend
• how do we ensure the offline git-bug identity of the user is the same as GitArena's created git-bug identity?

[cfkaran2]

I like where you're going with this, but I want to be sure I understand the code. Reading through the gitarena-issues crate, it appears you're operating directly on git-bug's internal database, correct? This may cause you issues down the road as it bypasses much of git-bug's internal architecture. My suggestion is that you use the GraphQL or the bridge interface instead; they are both designed as APIs, and so won't cause as many issues down the road. The rust2go crate may be helpful in creating the necessary bindings for this.

Thank you again for working on this!

[mellowagain]

I like where you're going with this, but I want to be sure I understand the code. Reading through the gitarena-issues crate, it appears you're operating directly on git-bug's internal database, correct? This may cause you issues down the road as it bypasses much of git-bug's internal architecture. My suggestion is that you use the GraphQL or the bridge interface instead; they are both designed as APIs, and so won't cause as many issues down the road. The rust2go crate may be helpful in creating the necessary bindings for this.

Thank you again for working on this!

yea that is correct. the reason i went for operating directly on the raw git objects is that there are issues with interfacing in other ways:

• directly calling the git-bug cli requires all self hosters to also install it and the goal of the project is to be as self contained as possible - for example we don't even require the git cli to be installed.
• calling the git-bugs Go library interface with rust2go (looks cool btw, didn't know this existed!) requires users to have the Go runtime installed and I don't really like FFI if it's not abstracted away, the goal of the project is #![forbid(unsafe_code)]
• the GraphQL api requires spinning up the git-bug api for every repository, which is too much of a burden in my opinion.
• i thought about just implementing a bridge that seems both just tacked on (and the goal was to be native to git-bug) and would be also something i'd have to discuss with git-bug authors as optimally we'd like to be a official bridge so it would ship it by default with the git-bug cli.

to prevent down-the-road issues, I setup tests in the gitarena-issues crate which are tested by the CI on every commit so we can catch issues early and fix them.

[cfkaran2]

Those are all very good points! I can see you've thought it through. That said, one of the reasons I brought up the issue was that I was concerned about possible races between git-arena and git-bug if they are both accessing the same repository at the same time. Do you know if git-bug has a file locking protocol that needs to be observed? I haven't read through either git-arena's or git-bug's code enough to know if there is a protocol that needs to be observed, I'm just paranoid about it!

Hmmm... this is me 'thinking out loud', so please forgive me for a half-baked idea, but would it be worthwhile to collaborate with the git-bug team to develop a spec for the bug database? That is, not only define what the schema for the database is, but also the access protocols. That would ensure that everyone that is accessing the database will do so in a well-coordinated fashion without stomping on each other's toes.

[mellowagain]

Those are all very good points! I can see you've thought it through. That said, one of the reasons I brought up the issue was that I was concerned about possible races between git-arena and git-bug if they are both accessing the same repository at the same time. Do you know if git-bug has a file locking protocol that needs to be observed? I haven't read through either git-arena's or git-bug's code enough to know if there is a protocol that needs to be observed, I'm just paranoid about it!

i don't think thats too much of a concern because really gitarena is thought to be on a long running server, meaning that no admin is also supposed to run the git-bug cli directly on the bare repo of a gitarena repo.

even if, we also just use libgit2 or gitoxide on gitarena which are required to by spec to implement the git file locking protocol as well so it shouldn't be a problem.

once I get to implementing this issues goal of allowing offline edits of git-bug issues, the CRDT should also handle any conflicts afaik.

Hmmm... this is me 'thinking out loud', so please forgive me for a half-baked idea, but would it be worthwhile to collaborate with the git-bug team to develop a spec for the bug database? That is, not only define what the schema for the database is, but also the access protocols. That would ensure that everyone that is accessing the database will do so in a well-coordinated fashion without stomping on each other's toes.

this is indeed a very interesting point, i have mostly just been relying on the git-bug docs and their source code for the implementation: https://github.com/git-bug/git-bug/blob/trunk/doc/design/data-model.md

i do think it'd be cool to have it standardized, i'll see if i can get in touch with anyone there to discuss it

[mellowagain]

i was able to get in touch with the git-bug authors and they added a official spec as well as some test data so I'll look into getting fully in-line with the spec and using the provided test data in our CI test flow

[cfkaran2]

i was able to get in touch with the git-bug authors and they added a official spec as well as some test data so I'll look into getting fully in-line with the spec and using the provided test data in our CI test flow

That's fantastic news! That'll solve all of my concerns in one go.

Also, just to address this comment:

i don't think thats too much of a concern because really gitarena is thought to be on a long running server, meaning that no admin is also supposed to run the git-bug cli directly on the bare repo of a gitarena repo.

I understand what you're saying, and I suspect that I want to use gitarena in a way that you don't expect me to. I don't have, and don't want to have, a central server that hosts gitarena; instead, I want to run it on my development machine directly. That gives me the power of gitarena (e.g., conversations over particular lines in pull requests, etc.) on my local laptop. As a single developer, this isn't that big a deal, but it's potentially very powerful when you have a small group of developers that are highly distributed.

As an example, in my last job we worked on robots out in the field... where 'in the field' really meant that you had no internet, and your cellphone only worked if you got really, really lucky. So each time you went out, you lost access to the central server. However, since half the team was in the field debugging, we were still writing code, tracking bugs, etc., that needed to be shared with the rest of the team. That's where git-bug came in handy; we didn't need to track the bugs in ad-hoc text files that we then logged when we were back in the office. Unfortunately, it still meant that we had to explain in the bug report where a given bug was likely to be located. GitHub (and I'm hoping gitarena) make this easy; they allow you to have a conversation while showing the relevant lines of code. That's really what I'm after; a truly personal GitHub that 'magically' synchronizes with anybody else that's nearby (git pushes via libp2p? Git bundles distributed via FLUTE over UDP? Other?) I hope that all makes sense to you. I know it's a niche application, but that's what really attracted me to gitarena in the first place; it looked like a truly decentralized GitHub.

[mellowagain]

thank you, interesting input. i'd say right now this is probably not a focus, also because gitarena is simply focused on the server-side design with also heavy dependencies on postgres and optional ones like zoekt or meilisearch.

The plan for decentralized forges is to implement ForgeFed/AT Proto (#74) but I understand that this doesn't address your concern of being offline first. as MR are not yet implemented, I am open to implementing it if there is a similar project like git-bug for it where it stores the convo and reviews within git refs, I think its a really elegant solution.

[cfkaran2]

I agree that storing the conversation, reviews, and the rest of the 'stuff' with a MR within git-bug's database is a truly elegant solution. In my mind, it's actually more elegant than trying to implement the ForgeFed/AT protocol as it lets git handle the server/data distribution part. Looking at it, the main use for ForgeFed/AT would be as a way of interchanging between things like git and mercurial or fossil. As long as you stay within the git universe, git-bug format extensions should handle the data, while git can handle data transport, synchronization, etc.

...if there is a similar project like git-bug for it where it stores the convo and reviews within git refs, I think its a really elegant solution.

I'm afraid I don't follow; now that git-bug's spec is open and has the ability to be extended, why not just define a new spec that builds on the git-bug format? Or am I missing something obvious???

[mellowagain]

I agree that storing the conversation, reviews, and the rest of the 'stuff' with a MR within git-bug's database is a truly elegant solution. In my mind, it's actually more elegant than trying to implement the ForgeFed/AT protocol as it lets git handle the server/data distribution part. Looking at it, the main use for ForgeFed/AT would be as a way of interchanging between things like git and mercurial or fossil. As long as you stay within the git universe, git-bug format extensions should handle the data, while git can handle data transport, synchronization, etc.

...if there is a similar project like git-bug for it where it stores the convo and reviews within git refs, I think its a really elegant solution.

I'm afraid I don't follow; now that git-bug's spec is open and has the ability to be extended, why not just define a new spec that builds on the git-bug format? Or am I missing something obvious???

yea i do agree storing the data in the actual git repo as objects is the truly elegant solution. i'll message the git-bug authors on their element about including MR support. feel free to chime in on the discussion too.

i saw you sent over a collaboration request for your git-bug fork but id rather not accept it as GitHub has some side effects of being a collaborator:

Owners of git-bug-spec will be able to see:

Your public profile information
Certain activity within this repository
Country of request origin
Your access level for this repository
Your IP address

especially because of the IP address thing id rather not, but I appreciate the invite.

regarding forgefed/ATProto support, I still think its a important feature to have not just because of portability (as that is correctly worked through with the git objects thing) but to allow profiles/issues/MRs to sync and be collaborated on across instances without having to setup some auto sync or manually --mirror pushing the repo to another instance. my vision is that something along the lines of the fediverse where many people will be on a certain instance but should not be bound to it.

the goal is to have the git-level object database in addition to federation support.