Thoughts on community testing for Particl Desktop 2.4.0.
here’s what i think would bring some structure to our upcoming testing:
Establish a public list of features to be tested
Since the new mp is supposed to have quite a lot of new features, it would be helpful to have a list of all the aspects of the mp that we can/should test together. This way everyone has a rough guideline of things to look out for.
Pair up and arrange specific testing sessions
In order to remove any uncertainty about transaction or reaction times between buyer and seller, people should make appointments and agree beforehand on the following:
- time, date and duration of the trading session
- who will act as buyer and initiate the buy/sell-flow
- specific features to be tested (e.g. buying from a private storefront, rejecting a bid, updating the inventory, …)
- channel of communication during the session
Document each session in a “test card”(*) and gather the data in one place
We should put together a simple standardized check sheet (test card) that everyone can use, where we document the conditions, findings and outcome of the session:
- operating system used
- features that were tested
- success or failure of the test
- processing and transaction times (stopwatching the key processes in the buy/sell-flow)
- uploaded image size of the items
- hickups or unexpected behavior
- comments on the UI or suggestions for improvement
This information should then be gathered in a central place, probably right here in the forum, so that the results can be analyzed and inspected by everyone involved.
(*) in the meantime I’ve created a test card in the form of a simple OO-spreadsheet… that I had to convert to .xls in order to upload it here, sigh. Have a look and let me know what you think, which fields to add etc. and if the formatting is broken now, LOL. I think we should keep it as simple as possible though. Otherwise no one is going to use it.
Give our feedback to the devs in an organized manner on discord
This one will will follow naturally from the points above but is still worth a mention, I think:
Before the launch of the mp, the testing-discord was a little messy in the sense that everyone seemed to state their individual findings as they went along. Maybe this time we can just write a short note after a testing session and refer to the sheets uploaded here.
Cheers and stay healthy
Those are very good points btmr is mentioning.
I fully share this
Additionally to the trading session (time schedule) we can use the green/red availability buttons on each ones profile to signal whether someone is available for testing. Others then can contact him by chat and then they can start testing.
“Before the launch of the mp, the testing-discord was a little messy in the sense that everyone seemed to state their individual findings as they went along. Maybe this time we can just write a short note after a testing session and refer to the sheets uploaded here”.
Excellent suggestion right there going to make it a whole lot easier for our devs to sort thru everything.
Sounds like a great path forward. Nice work!!
First of all, I am new to this community. I’m Sylvain & from Canada. I have only been in crypto since 2018, so fairly new, but I am passionate about it. When I heard of Particl, I was like, I would like to contribute to this, help out etc… If what I say does not make sense, especially when it pertains to the coin/marketplace itself, don’t hesitate to let me know lol. I have been reading and learning in the background, so will improve my knowledge of the project for sure.
Here are some of the topics we talked about:
Document each session in a “test card”(*) and gather the data in one place
We like the idea of this checklist (excel sheet). that way it gives a standardized format to keep track of the different testing.
We also thought of a possible new “status” color…say blue…to keep track of those who are online and that could easily interact with other people actively conducting testing. @ffmad is this even possible?
GitHub vs. Other social medias
There was a discussion over the various medias today (riot, telegram/discord) on how we would conduct the fault reporting. It was passed on that the preferred method would be through Github. If this is the adopted plan, we could report our issues to the forum/ spreadsheets and have one person report directly to the GitHub to facilitate the work of developers.
This was mainly brought up in an effort to keep the process simple and allow crypto newbies to either report and/or down the road being able to utilize the marketplace.
In order to facilitate the scheduling of testing, I will start a new topic so everyone interested in helping out with the testing, can give their region/timezone and availability. That way, we will have a better idea where people are and prepare ahead of time.
Hope it makes sense! If there are other points/topics not covered, keep bringing it up in this forum. We think the forum might be the best place to discuss everything about the testing and developments (more business oriented).
Have a great day!
that gives us the following TODO-list then:
1) Assemble a list of features to put in the “Tested Feature” column in the test card
For general functionality to be tested, I think everyone should just pitch their suggestions in here and I’ll include them. When in doubt, just fire up the current mainnet-release as a reference.
For the new features (Infinite Markets, Market Management, Identity Management, Seller Profiles etc.) we either need to probe the devs or simply wait for the official reveal.
Action: everyone, btmr
2) Settle on a communication-channel during the testing sessions
My suggestion is to just use direct messaging on Discord. This way we won’t clog up any space in the forum or in the particl-Discord. Please chime in with agreement or other suggestions here.
3) Agree on the process-flow for bug reporting
In my mind, this is where we arrived at now:
- Test results & findings are documented through testcards
- Testcards are uploaded to a common forum topic for everyone to see and discuss
- Community then filters for legitimate bugs or unexpected behavior
- Selected community member (*) files an issue on github
4) Appoint one or more members to take care of filing the issues on github
I’ve done it before and I would volunteer to do that. Please tell me if you’re ok with that.
Still, we would need at least a second person to handle it, in case I’m not available at the time or if we just find a lot of bugs
Action: btmr, everyone
5) Scheduling: collect availability/regional infos for testing in a separate forum topic
Everyone ok with this?
Hey @btmr, sounds great!
Maybe we can even use this kind of testing environment later on the mainnet as well. When someone will find out some kind of misbehavior on mainnet, others can set up some testlistings and then try to reproduce and communicate this error/bug on github.
To your TODO-list:
- On the current mainnet the proposal section often shows some kind of irritating voting results. I would suggest to add this as testing feature as well
- For direct messaging we also could use the forums “chat” function (speech bubble on the upper right). By that we would stay focused on the forum (apart from github).
- I guess testcards also being uploaded when no bug has been found, isn’t it? For making it easier to filter bugs, shall we ask the tester also to mention separately with the posting when he discovered a new bug. This could make it easier for others to filter the testcards.
- btmr you’re the best for this Thx!
- I think every tester should at least check once per day his listings for potentially incoming bids. Best would be to have direct contact with the seller/buyer. However, sometimes it would also be interesting to check whether smsg-messaging also works reliably when not both wallets are online and unlocked at the same time.
Is there a definition when something can be described as a bug or minor malfunction?
What about making a kind of competition with the devs If we find less then 2 bugs/malfunctions we are obliged to provide something funny, if we find more than 5 bugs/malfunctions it’s the devs turn.
That’s just an idea. Also wouldn’t have a problem if we just leave it as it is without competition.
@ 1: Will do.
@ 2. Oh that’s cool too! I didn’t know about this feature.
@ 3. Yes, we need to share successful tests as well. A bug may appear only with a specific OS or only from time to time. It’s just as important to get stats on what is working well.
@ 4. Aye.
@ Yes, the participants should organize “boundary conditions” like these before the testing session. I will think about a way to include info like this in the test card.
Good question. Here’s my take on this:
To me, a “bug” is any behavior of the system that leads to either:
an incorrect result,
no result when there should be one,
a completely unexpected result that the user had no chance of anticipating.
Sticking to this “definition”, I would not make a difference between minor and major bugs.
There may be “unexpected behavior” that just needs a little more explanation and is otherwise fine.
Here’s where it helps to discuss on a community-level before bringing it to the devs.
Regarding the competition: I’m game for anything that gets people motivated to fill out spreadsheets in their spare time…
Your bug “definition” sounds very reasonable.
For this testing round I set up a small table on the various time zones.
I assigned a letter to each time window of 1 hour. Maybe we could add to each listing as title/description an identifier (eg V001) to know who set up those listing and at around what time he/she will check on a daily base his listings for bids and orders.
Let’s say for this testing round in April 2020 we could assign a typical “Product Name” for a listing as follows:
By that a potential buyer could see that it’s V001 who is selling the item and that he typically will check its listings between 8 am and 9 am Pacific Time (P ) for incoming bids. For contacting the seller, the buyer could look up in a “VolunteerList” we could set up as topic on the forum and where every volunteer of the testing round can put a post in, where he defines its identifier and maybe also his prefered times where he typically will check his listings. Since the creator of this posting can always edit his posting, he afterwards can still change his entries. (three dots on the lower right of its posting)
Instead of a pseudonym like V001 (eg. for volunteer nb. 1) we could also use our profile names, but I thought for later smaller testing rounds (e.g. on mainnet as well), not everybody would like to have his profile name publicly visible on the marketplace.
For letting the seller to know who the bidder is, the bidder should place its identifier into the shipping address.
This could be done like that:
First Name: Test-Apr20
Last Name: V001 (or other identifier)
Address 1: P (typical daily check of my listings/bids)
That’s a great idea to have identifier codes for the testnet-items.
I say let’s just do it like this and see how it goes!
FYI: test card now updated with a range of pre-selected features taken from current client.
yeah, you’re right. I should have posted the example in here straight away.
The volunteer list should have only be meant as for listing who is taking part in the testing and what ID and time preferences he has (points 1 - 4). Based on this information it should be easier to contact certain seller/buyer in case of communication needs.
The nice thing on the forum is, that you can update (edit) your post. So every volunteer needs only one entry in the volunteer list topic and provide there in some short lines some information on his seller/buyer profile. I also wouldn’t post listings or other comments in there, to keep it as simple as possible to brows through the participating volunteers.
What do you think about uploading the test cards. Shall we do this also in there, or in a separate topic. Shall we upload a .xls file, or open office .ods for each day separately, or shall we make one test card file per volunteer which contains separate sheets for each testing day?
So I deleted my remarks in volunteer list and pasted them back in here:
Hey @Sylvaincloutier @btmr, what do you think when we put the identifier on first topic and we don’t use telegram or discord. By making a post and stating each individually choosen identifier (e.g. PK01) there’s always a way to contact the “tester” on his forums profile. When needed they still can exchange their telegram/discord/riot username. After each testing day, the tester can upload another testcard to his post. So every tester only has to make one post inside this topic and can afterwards edit its data and uploading further testing cards (.xls-files).
When making a listing or bid on particl marketplace the data to enter might look like as follows:
Example for Seller Data:
Product Name: PK01-P Porsche911
Short-description: Car for fun … xxx
Example for Buyer Data:
First Name: PK01
Last Name: P
Address 1: xxx
Address 2: xxx
@Pancake I agree in keeping the volunteer list clean. We should probably start a new topic with the test cards in my opinion, that way we keep each subject separate. Thanks for your input again.
In regards to the identifier it was my fault to come up with PKxx
I think initially I proposed to use V001 where V could stand for Volunteer. When Sylvain mentioned to use maybe the user name of telegram/discord, I thought maybe to leave it up to everyone to choose it’s own identifier, … so I came up with PK for Pancake
So PK means nothing special. Then we rather should use V… for volunteer or any other suggestions? We can also leave it up to everone to choose its own identifier. For me both ways are ok.
Yeah that’s fine. Let’s start another topic for the test cards as well. Maybe let’s choose for both topics (test cards and volunteers) a similar title. Thus it gets clear they belong together. Maybe it also makes sense to set up a third topic like “bug reports”. Apart from the forums personal chat function we can use this topic to discus and answer potential questions to any bugs which have been reported within the test cards.
Just as a proposal for the three topic titles:
- 2.4.0-Testnet - Volunteer List
- 2.4.0-Testnet - Test Cards
- 2.4.0-Testnet - Bug Reports
Ha, no sweat.
Yeah let’s just use our username (or an abbreviation thereof) then. It makes things a little easier.
I agree to have these three topics, so we know what goes where.
Maybe switch places like that: “Volunteer List - 2.4.0 Testnet”. This way, you immediately see where to go, when there’s a new message notification.
- Volunteer List - 2.4.0 Testnet
- Test Cards - 2.4.0 Testnet
- Bug Reports - 2.4.0 Testnet
and everyone can choose the identifier he likes