(OT) Tracking Issues, Requirements, Tests

afoken has asked for the wisdom of the Perl Monks concerning the following question:

Hi!

I'm searching for a sane tracking software, for developing medical devices. Let me explain:

At work, we are currently using a really old version of Jira, with add-ons for tracking requirements and product tests. It's so f*ed up that all attempts to update it fail. Updating was never easy, but the last available installer/updater package actively destroys configuration and database, rendering the installation unusable. The fine print: You need to be at the latest version to be able to migrate to the cloud Jira. We can't get to that version. Tech support is not helpful.

Now, we running one project in a cloud Jira setup, paid for by our customer. I always thought the user interface of the old server installation was bad, but the cloud user interface is a mine field. Ideally, you run it on a separate computer, with mouse and keyboard disconnected. Only connect them if you really want to interact with it. Because almost any key and any click anywhere on the user interface triggers some action, even in empty spaces. The screen is littered with junk that you don't need, stuff that you need is well hidden. And as if all of that was not worse enough, we completely give up control of the data we enter into the system. Atlassian as a long history of mishandling or deleting user data, and we legally have to keep that data. I don't see any sane way of doing that.

So, guess what? I want to get rid of Jira. I don't want to even have to think about migrating existing data from our existing Jira instances to cloud Jira, and I don't want to use cloud Jira. And as much as I would like to, I don't want to write my own system. I want an existing software, running on a local webserver. I would prefer open source software, so we could tweak things if we really need to. I don't think we need much tech support, so a "community support for free, pay for professional support" model (as for Proxmox or Virtualbox) would be ok. We could pay a license fee, but we are a small company. We mostly do small projects, so license fees that costs in the order of an entire project are simply impossible.

What do we need?

local installation (no cloud)
web-based
runs on Linux
preferably open source
issue tracking
requirement tracking
tracking test plans and test plan execution (product validation, testing that you fulfilled all requirements)
a way to import old (non-cloud) tickets from Jira, including how they are linked - a REST API might be sufficient, so I can scrape data from the existing Jira instances

The classic Bugzilla ticks almost all checkboxes, except that it can't do requirement tracking and test tracking. And it's name is burned because an unloved and unmaintained installation lingered around since well before I was hired. That's clearly not the fault of Bugzilla, but some people are just strange.

Before Jira, we (the software developers) starting using Redmine just for tracking issues, with the intention to make the entire company using it. Jira won because Redmine wasn't even invited to the race. Redmine has no requirement and test tracking, but at least it has interfaces to third party tools.

A final note: Jira (with pluins) integrates the bugtracker on the one hand and requirement / tests tracking on the other hand into a single software. We are used to it, but we don't need that integration. Some bug tickets may refer to requirements or tests, but that's all. There is no tracking, not even counting. Nobody checks how many bugs a requirement "attracted". It is convienient to just type FOO-123 and it will be converted to a link to a requirement, just like typing FOO-456 will be converted to a link to an issue. But honestly, copying https://reqs.example.com/req-tracker/req/FOO-123 from the browser's address line into a issue ticket would hurt nobody.

So, how do you track issues, requirements, tests? Do you know a software that might fit our needs?

Alexander

--
Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

Comment on (OT) Tracking Issues, Requirements, Tests

Replies are listed 'Best First'.
Re: (OT) Tracking Issues, Requirements, Tests by Corion (Patriarch) on Apr 07, 2025 at 09:36 UTC
There is the OTRS family of ticket trackers. I've only used them as a customer, but I think you can do at least requirement tracking with them as well. I'm not sure whether they have a workflow for tests (test definition and executions of the test definitions) though. OTRS Pro (closed source) OTRS community edition OTOBO - fork of the last open version of OTRS, professional support Znuny - another fork, also professional support In the medium run, Jira has an OpenAPI spec so you it is relatively easy to synchronize/import data into Jira if you can get your data into the representation of the Jira schema.	[reply]
Re: (OT) Tracking Issues, Requirements, Tests by karlgoethebier (Abbot) on Apr 07, 2025 at 10:44 UTC
In the last company I was with they used some OTRS based solution for tracking tickets: Amos Support. Amos is a MRO software solution for airplanes used by a variety of airlines. ŤThe Crux of the Biscuit is the Apostropheť	[reply]
Re: (OT) Tracking Issues, Requirements, Tests by NERDVANA (Priest) on Apr 08, 2025 at 23:25 UTC
We have one customer on GitHub, one on GitLab, and one on Gitea. All 3 of these work quite well for our size of dev teams (small). GitHub is out if you don't want a cloud, but the other two can be self-hosted. None of them separate issues from requirements from tests, but they all have numerous ways to tag and organize issues, which seems to work fine in practice. (I also don't understand why you'd want tests organized like issues... why not just create an issue for adding a test, write it and add it to the automated test suite, and then resolve the issue? If it's a documentation thing, then just use the wiki.) As it happens, the largest customer is the one using GitLab and they are considering Jira now because GitLab is highly developer-focused and most of their employees are non-developers, and price for accounts is fairly high on enterprise GitLab, so they can't just give everyone an account. Meanwhile, the second largest customer is using the free version of Gitea and integrated it with Active Directory, so all employees automatically have an account if they need to interact with developers.	[reply]
Re^2: (OT) Tracking Issues, Requirements, Tests by afoken (Chancellor) on Apr 09, 2025 at 21:40 UTC
why not just create an issue for adding a test, write it and add it to the automated test suite, and then resolve the issue? Because we track the entire product, including its documentation, electronics, mechanics, pneumatic (if any). There are tests that can not be automated, at least not for a sane price. One of our current projects has a container for some liquid, with a level sensor and a tilt sensor. The container has a manually operated draining valve and - for development and testing - a simple funnel on the top side. (A little bit like the fuel tank on old motor bikes, but for a completely different purpose.) I don't know the exact test spec yet, but I bet you will find instructions like these in the test plans: ... Fill ten volume units of liquid into the container Wait three time units Check the if the volume displayed by the device is between 9.5 and 10.5 units. Tilt the container by five angle units. Wait one time unit Check if the device issued a tilt alarm Return the container to level Wait one time unit Check that the alarm has turned off Drain five volume units out of the container Wait one time unit Check that the device issued a leakage alarm ... Yes, these are very detailed baby steps of actually using (or misusing) the device under test. Using your fingers, ears, eyes, and a little bit of that grey filling material between the ears. Pretend to be a smart or dumb user. Yes, you could automate that, using a machine three times as complex as the device under test. Or just tell an intern to open the test plan #47893 in a browser and to follow the instructions. These tests are usually executed and documented ONCE for the entire device, before handing it over to the client. The test instructions and results are part of the documentation that is handed to the client. Maybe after year or two, hardware and/or software are modified to better match the client's needs, tests are modified to match the new requirements, and then, those tests are run ONCE to confirm that all requirements are still fulfilled. So even just thinking about automating them is way too expensive. Interns are way cheaper than automating those tests. Even if they completely f*** up a device under test. Another part of the tests is simply reading the source code (or schematics, layouts, hardware drawings). Compiler warnings are great, and lint find a lot of extra mess-ups, but having another developer look at the code (or schematics, plans) can find a lot of those nasty edge cases everybody hates to debug. Alexander -- Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)	[reply]
Re^3: (OT) Tracking Issues, Requirements, Tests by NERDVANA (Priest) on Apr 10, 2025 at 02:21 UTC
Ok, so not software tests. Well, then that goes to my point about the wiki- each of these platforms has a version controlled wiki and you could easily set up a standard location for all real-world testing plans to be documented. If you want them in the same repo as the code, then just designate a subdirectory for it and write them in Markdown, which renders nicely and is easy to edit from the web interface.	[reply]
Re^4: (OT) Tracking Issues, Requirements, Tests by afoken (Chancellor) on Apr 10, 2025 at 07:45 UTC
Re^2: (OT) Tracking Issues, Requirements, Tests by cavac (Prior) on Apr 10, 2025 at 12:54 UTC
add it to the automated test suite As afoken already said, in big real world applications, you often can only automate a small amount of tests. For my point-of-sales system, about 50% of the issues in the field are usability issues of the UI. And yes, that includes things like "an old lady refusing to wear her glasses and complaining that the text is too small" (so we added scalability to the main UI mask texts and buttons), touch screen gestures getting misinterpreted because the user did do them in the laziest way possible... A lot of UI stuff is very subjective and the results vary from customer to customer and from user to user and from device to device. Yes, one may (and should) write tests for some of those cases, when it is the best financial choice. But in many cases, a trained eye can catch errors in seconds or minute for which you would have to spend weeks writing test software. And you would need to keep spending weeks writing updates to the software for every small UI change, instead of going through a 10 minute checklist manually... PerlMonks XP is useless? Not anymore: XPD - Do more with your PerlMonks XP Also check out my sisters artwork and my weekly webcomics	[reply]
Re: (OT) Tracking Issues, Requirements, Tests by trwww (Priest) on Apr 07, 2025 at 16:21 UTC
Request Tracker is quite popular: https://github.com/bestpractical/rt https://bestpractical.com/request-tracker	[reply]
Re^2: (OT) Tracking Issues, Requirements, Tests by jdporter (Paladin) on Apr 07, 2025 at 18:36 UTC
I'd like to recommend RT... but frankly I don't think it's awesome. More to the point, it does issue tracking but seems to be missing the other two features the OP wants — requirements tracking and test tracking.	[reply]
Re^3: (OT) Tracking Issues, Requirements, Tests by trwww (Priest) on Apr 09, 2025 at 12:39 UTC
Are there industry standard definitions of "requirements tracking" and "test tracking"? Is there lists of features for each tracking type that must be available before you can call it such? Extending request tracker is very, very easy, though I admit thats easy for me to say because I've been paid to do so in the past. It doesn't ONLY do request tracking, it does just plain tracking, of anything you feel like tracking with it.	[reply]
Re^4: (OT) Tracking Issues, Requirements, Tests by Corion (Patriarch) on Apr 09, 2025 at 12:43 UTC
Re: (OT) Tracking Issues, Requirements, Tests by afoken (Chancellor) on Apr 08, 2025 at 08:11 UTC
Thanks a lot, thats a good set of software to look at. Alexander -- Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)	[reply]