Feedback and Suggested Improvement to Bug Bounty Reward System

akincibor 2024年12月7日

Dear,

I hope this message finds you well. I am writing to express my concerns about the current reward distribution model for the bug bounty program and to propose a more effective alternative that benefits both researchers and the program’s objectives.

The existing system, where the monthly reward pool of 1000 $ABT is divided equally among all valid submissions, unintentionally discourages researchers from reporting multiple findings. For instance, if I submit two valid reports, my reward per report is halved compared to submitting only one. Furthermore, if other researchers also submit findings, the rewards per submission are further diluted, reducing the overall incentive to report multiple issues.

In my case, I have identified 20 additional issues but have chosen not to report them this month because doing so would drastically reduce my reward per finding to an insignificant amount. I suspect that many researchers adopt a similar strategy, reporting only 1-2 bugs per month to avoid diluting their rewards. This approach hinders the program’s effectiveness, as critical issues may remain unreported for extended periods.

To address these concerns and promote more robust participation, I suggest implementing a reward system that combines fixed payouts based on bug severity with an additional monthly bonus pool. Here’s how it could work:

Fixed Rewards Based on Severity:

Low severity: X $ABT per valid finding
Medium severity: Y $ABT per valid finding
High severity: Z $ABT per valid finding This ensures fair compensation for researchers regardless of the number of participants or submissions.

Monthly Bonus Pool: A bonus pool of 1000 $ABT can be distributed among researchers based on criteria such as:

Total number of valid submissions.
Impact and criticality of findings.
Contributions to improving the program, such as suggesting fixes or participating in discussions.

This dual system would guarantee fair rewards for individual contributions while incentivizing researchers to report more issues without fear of diminishing returns. It would also encourage higher participation and faster resolution of vulnerabilities, making the program more effective overall.

Thank you for taking the time to consider this feedback. I am happy to discuss these ideas further or assist in developing a more equitable reward structure that aligns with the program’s goals.

Best regards,

20 条回复

akincibor21个月前

But the reward pool is 1000 ABT so if I report 10 fatal bug in a month or 100 fatal bug in a month or 1000 fatal bug in a month, the reward pool is still 1000 ABT. In this case my reward will be weighted but cannot exceed 1000 ABT. So compared to someone that reported a 1 fatal bug and another one 1000 fatal bug, due to weighted bounty someone will earn 999 ABT and the other one 1 ABT because the pool max reward is 1000 ABT

Brain Titan21个月前

so your suggestion is to increase the pool size? That is reasonable, I also suggested to the team that 1000 ABT for Bug Bounty is a little low

akincibor21个月前

This is not fair due to the fact that the pool is capped to 1000 ABT

in a month where there is only 1 minor bug that is qualified = 1000/ (0.5/0.5) = 1000/1 = 1000 ABT

in a month where 1 reporter reported 1 fatal bug and another one 1 fatal bug too = 1000/(100/50) = 1000/2 = 500 ABT each reporter

So someone could earned 1000 ABT in a month for a minor bug while in another month someone will earn 500 ABT for a fatal bug.

That's why I'm currently holding 20+ high critical bugs for the next month because this is not fair

Brain Titan21个月前(edited)

I get you. Your issue with the current system is that it is disadvantageous for people with a high quantity of fatal bugs per month, coming to a point that if you report too many of those, your reward is “capped” and has diminishing returns in terms of reward ratio

In that case, yes, calculate how many of those fatal bugs are worth reporting that month to get the maximum ABT from the pool without getting to a point where you “wasted” bug reports due to diminishing returns.

This will depend a lot on how many other users contribute to the Bug Bounty program that month and how critical their bugs reported are. With fewer contributors, you can afford to only submit a few fatal bugs monthly while getting the max reward you can, while with more contributors that month, you will have to submit more fatal bugs to get the same reward.

In such scenarios, it makes sense to save some fatal bugs for months to come

Akincibor21个月前

By the way, we were talking to each other's locked report haha. The platform is full of bugs and if you want us to report each of them as fast as possible you need to fix this dilution problem

Akincibor21个月前

Yeah, don't worry if they do not fix it I'm not going to report other bugs this month. There are you and me currently reporting security issues so we can collab or I can wait you to report 3 bugs on January and I report also 3 bugs next month.

Abir Khan21个月前

Ok. We collab .

Come on inbox

Robert21个月前(edited)

To make it more clear and easy to follow, I used Chat GPT to improve it:

Understanding the Principles of Our Bug Bounty Program

Thank you for the insightful discussion on enhancing the bug bounty program. I’d like to take this opportunity to outline the principles behind its design and the rationale driving our approach. These principles aim to foster meaningful participation and discussions within our community while continuously improving the system’s quality.

The Concept Behind the Program

Our bug bounty program is inspired by the reward mechanism used in Bitcoin’s mining system. In Bitcoin, block rewards are fixed and diminish over time through halving events. Similarly, the bug bounty rewards are set at 1,000 ABT per period, with plans to gradually reduce the rewards over time.

The key aspect here is that the fiat value of the rewards is not predetermined but rather depends on the market price of ABT. When ABT’s fiat price is low, the fiat value of the rewards is proportionally low, and when the price rises, so does the reward’s value. This dynamic reflects the inherent relationship between the system’s quality and its token’s value. A higher-quality system—one with fewer bugs, better security, and an improved user experience—can lead to an increase in the token's value, benefiting all holders.

Participation as a Game Theory Exercise

Engaging in the bug bounty program mirrors a strategic game theory exercise where participants must weigh their actions based on both the program’s structure and the behavior of others:

Competition vs. Collaboration With fewer participants, each individual stands to gain a larger share of the rewards. As more participants join, the rewards are divided among a larger pool. This dynamic encourages healthy competition while ultimately driving improvements in the system's quality.
Improving System Quality As bugs and issues are resolved, the system becomes more robust. Over time, it becomes increasingly challenging to find new issues to report. This naturally raises the stakes for participants and aligns incentives with improving the ecosystem.
Strategic Decisions Participants must decide how and when to submit their findings. Should one submit all discovered issues at once or stagger them over time? This decision depends on personal strategies and the actions of others, making each participant’s approach unique.

Motivations for Different Participants

The program accommodates a diverse range of participants, each with their own motivations and strategies:

Token Holders Token holders often provide feedback with the broader goal of improving the system’s quality. They may be less concerned with immediate financial rewards because a better system inherently increases the token’s value, benefiting them directly.
Non-Holders Non-holders may view the program as an opportunity to assess the token's fair value or earn financial rewards. Some may choose to hold onto their tokens after participating, while others may sell their rewards if they lack long-term interest in the system.
Active Contributors For participants who identify multiple issues, timing becomes an important factor. Should they submit all findings at once or spread them out to maximize their chances of earning more rewards? （though it's unlikely they can increase chance) The program’s structure leaves this decision to the participants, ensuring flexibility.

A Holistic Approach to Quality

While security is undeniably important, it is not the sole focus of the program. The quality of the system is influenced by numerous factors, including security, user experience, and overall reliability. For this reason, we will not create a separate reward pool specifically for security issues. Instead, we take a comprehensive approach to improving the system, addressing issues proactively, whether they are submitted by participants or identified internally.

Closing Thoughts

Our bug bounty program is designed to be dynamic, inclusive, and aligned with the long-term interests of our community and token holders. By participating, contributors not only help improve the system but also engage in a rewarding process that reflects the interplay of strategy, quality, and value. As we continue refining the program, we remain committed to fostering collaboration and ensuring the system’s ongoing success.

akincibor21个月前

Thank you for your response, but it seems there is still a misunderstanding. My concern is not about the price of your token or the dollar value of the bounty. My issue lies with the dilution of the reward when reporting multiple bugs.

Why would I report extra bugs, knowing that for each additional bug I report, my bounty will be punished by being diluted? The current system disincentivizes thorough participation because every extra report I submit reduces the reward per bug for myself and everyone else. This is counterproductive and ultimately harms the program's goal of improving system quality.

If this structure remains unchanged, I see no reason to report the remaining bugs I’ve discovered—currently over 30 issues, ranging from low-severity to fatal vulnerabilities.

Please reconsider the reward system to encourage researchers to report all findings without fear of being penalized for their contributions.

Robert21个月前

z1emY7syMxHYQgEWr35xku3J2oLoPnD4jL8 I asked ChatGPT to help you analysis your strategy how to submit the issues to maximumize your reward, here is what she said:

Based on the principles outlined above, here’s an analysis of the pros and cons of submitting all discovered issues at once versus submitting them in batches:

Submitting All Issues at Once

Advantages:

Ensures Priority If the reward mechanism prioritizes a first-come, first-served basis, submitting all issues at once ensures you claim rewards for all your discoveries, avoiding the risk of others submitting them first.
Simplified Process Submitting all issues at once reduces the workload of repeated submissions, allowing you to focus on completing your reports.
Higher Rewards in Low Competition Periods If there are fewer participants in a specific period, submitting all issues at once can secure a larger share of the fixed 1000 ABT reward pool.

Disadvantages:

Reward Pool Cap Since the reward pool for each period is fixed, submitting a large number of issues at once might dilute the reward per issue, reducing individual payouts.
Missing Future Opportunities As the system’s quality improves, discovering new issues will become more difficult. By submitting all issues at once, you might miss opportunities to participate and earn rewards in future periods.

Submitting Issues in Batches

Advantages:

Risk Distribution Submitting issues in batches spreads the risk across multiple periods, avoiding reliance on a single reward pool.
Adapting to Competition If a period sees heavy participation, holding back some issues for a less competitive period can maximize rewards per issue.
Sustained Participation Batch submissions allow you to remain active across multiple periods, giving you more opportunities to observe others’ behavior and adjust your strategy.
Responding to Price Fluctuations Since the fiat value of rewards depends on ABT’s price, batching allows you to submit more issues during periods when ABT’s price is higher, increasing your actual reward value.

Disadvantages:

Losing Priority If other participants submit issues you’ve held back, you may lose the opportunity to claim rewards for those issues.
Increased Submission Costs Batch submissions require more time and effort, potentially adding complexity to the submission process.

Comprehensive Recommendation: Choosing the Right Strategy

To decide between submitting all issues at once or in batches, consider the following factors:

Competition Level If the current period has fewer participants, submitting all issues at once may be advantageous. If competition is high, batching can prevent rewards from being diluted.
Issue Quality and Quantity Submit high-severity and impactful issues as early as possible to secure rewards. Hold back less critical issues to distribute submissions over time.
ABT Price Fluctuations If you expect ABT’s price to increase in the future, saving some issues for later submission can result in higher fiat rewards.
Long-Term Participation Goals If you plan to engage with the bug bounty program over the long term, batching submissions ensures consistent activity and earnings. If your goal is short-term maximization, submitting all issues at once may be more effective.

Recommended Strategy: Stay Flexible

The best approach is to remain flexible and adapt to circumstances. For example:

Prioritize submitting critical issues to secure rewards.
Submit less critical issues in batches to maintain long-term participation.
Monitor other participants’ behaviors and adjust your strategy accordingly—for instance, holding back some issues during highly competitive periods.

This dynamic strategy not only optimizes your rewards but also ensures your participation remains strategic and adaptable.

shenxiuqiang21个月前

我认为安全漏洞应该提供额外的奖励，上不封顶。

shenxiuqiang21个月前

是不是我们理解的“安全漏洞”意思不同？

Robert21个月前

没有不同。是不赞同 “上不封顶“ 这种做法。

akincibor21个月前

No one is asking for an unlimited bounty. I think you still fail to understand how the current system impacts not just researchers like us but also your platform. Implementing a minimum fixed amount for security-related issues would be fair to researchers and incentivize us to report bugs as soon as we find them.

You’re going to pay for these bugs eventually, whether now or a year later. The only difference is that with a fixed minimum, you’ll get critical vulnerabilities reported sooner, improving your platform's security faster. It’s a win-win situation for everyone.

Robert21个月前

These issues won't be delayed for months or years, as they will be addressed by our community and team sooner or later. In the crypto industry, we must constantly combat all kinds of security threats, and we approach this with seriousness and strategic intent. A fixed price per issue is simply not how we operate.

akincibor21个月前

As a white hacker, my focus is on security vulnerabilities, serious issues like injections, leaks, account takeovers, database access, PII exposure, and remote code execution. These bugs, if exploited, could irreparably damage the platform. Meanwhile, I’ve seen multiple reports for non-security issues like buttons not working, 404 errors, or default language settings being flagged and rewarded from the same 1000 ABT pool. While those reports might be valid for improving user experience, security vulnerabilities deserve separate prioritization and rewards. Bug bounty exists to reward hackers for not exploiting the flaws they find, a fundamental pillar of ethical hacking. It needs to be attractive, fair, and focused. Right now, this structure doesn’t incentivize white hackers like me and Abir Khan to report impactful issues. Instead, it actively punishes researchers who contribute more by diluting their rewards with every additional report. And saying again, I’m not discussing here for the dollar value of the bounty. I don’t care if ABT is worth $1 or $1000. My concern is fairness.

null21个月前

固定奖金池挺好的 👍

👍1

akincibor21个月前(edited)

That’s how 99.99% of bug bounty programs work, my friend, everyone works for money. But let me clarify: my concern isn’t about the price of ABT, as Robert keeps mentioning.

I haven’t even been active on HackerOne for over a year because 90% of my time is spent on Immunefi. The minimum payout there for a low-severity bug is $1,000, and critical issues can go up to $10 million. So yes, I’m earning well there.

If I’m here working on ArcBlock’s platform for $100, it’s clearly not about the money.

Wasil21个月前(edited)

As I said, it’s the standard most are familiar with, but not the only solution, and not an absolute requirement.

While it is unattractive to some, it does attract others.

From my perspective, it’s a win no matter what you do.

But I can see that from your perspective, it appears unfair for your research to be rewarded at the same par as a minor issue. Or be diluted when you have many issues to address.

However, ArcBlock tries to be as fair as possible in my eyes, considering there’s not many places one can earn token rewards for simple bugs.

Where you see unfairness, 100 other people see their first chance to be a part of a community that recognizes the smaller members of its ecosystem as equals, requiring the giants to humble themselves, and work along side the normies, who’d they otherwise right off as fools.

To me, that’s fairness. To others, maybe not so much.

你是好人，我跟你17个月前

有意思的帖子，我收藏了