To find problems in Alatau City Bank's credit funnel, we invited 40 respondents. Analyzing 40 hours of video manually is time-consuming and expensive, so we delegated the routine to our internal AI service, Rocket Insight Lab. Here is how automation helped us find non-obvious infrastructure barriers in the product.
Project inputs
- Client: Alatau City Bank.
- Task: Identify the reasons for user drop-off during the loan application process in the app.
- Method: Quantitative moderated usability testing.
- Scale: 40 respondents, Almaty, 2026.
- Tool: Proprietary AI service Rocket Insight Lab.
What the client came with
The bank formulated a hypothesis: the loan application funnel in the mobile app was operating inefficiently. Dashboards recorded user drop-offs at specific stages. The client needed to confirm the hypothesis and get a prioritized list of barriers.
Why a quantitative method and a real product
Classic qualitative research with 8–12 people finds interface barriers. But it does not show the scale of the problem. Without frequency data, the team does not know which error to fix first.
We organized testing with 40 respondents. With such a sample, the frequency of a barrier becomes an argument for making business decisions.
Participants went through the scenario on their own smartphones in the release version of the app. This allowed us to account for device diversity, mobile connection quality, and lighting.
Participant screening
We selected respondents for a first-time loan application scenario. Sample requirements:
- Citizens of Kazakhstan from Almaty aged 18 to 65.
- Banking service users willing to consider lending.
- No active loans with the client bank.
- Positive credit history without delays.
- Willingness to submit a real application in the app without final execution.
A clean credit history was a key methodological requirement. Bank rejections should not be attributed to a "bad borrower" status. Participants went through the bank's actual scoring. This allowed us to test the most complex part of the funnel—the decision-making stage, which cannot be simulated on a test stand.
How the sessions went
We conducted 40 in-person sessions at the Rocket Tech UX laboratory. The respondent held their own phone. An external camera recorded the device screen, grip, and finger movements. The moderator did not prompt steps. They kept the participant in the scenario and recorded moments when expectation diverged from reality.
Rocket Insight Lab: AI pipeline for analytics
Analyzing 40 hours of video is the main obstacle to frequent quantitative research. Manual transcription, tagging remarks, and calculating frequency burn budget and deadlines. We delegated these tasks to the AI service Rocket Insight Lab.
How the algorithm works:
- Transcription with timecodes. Video is converted to text. The system separates remarks by speakers and links them to seconds on the recording.
- Primary tagging. The algorithm looks for candidate fragments: questions about the interface, pauses, comments about errors.
- Pattern clustering. AI consolidates similar observations from different sessions into unified groups. This eliminates the human factor—the algorithm does not forget phrasing from the first interviews by the end of the sample.
- Forming pain points. Patterns are assembled into descriptions of specific problems.
- Frequency calculation. The service counts in how many of the 40 sessions each problem appeared. This is the basis for fair prioritization.
- Traceability. Every statement in the report is linked to the original timecode. We documented 97 observations, any of which can be double-checked in one click.
The researcher's role
AI took over the routine but left decision-making to the human. The UX researcher validated clusters, removed false positives, described behavioral mechanics, and formulated product recommendations.
The data processing stage was reduced from an estimated 60 hours to an actual 12 hours. Analytics costs dropped by 80%. We invested the freed-up resources into the depth of analysis—uncovering and verifying cross-links between different barriers.
How we prioritized findings
A team cannot take on 11 barriers at once. We ranked them by two metrics:
- Frequency — how many of the 40 participants encountered the problem.
- Criticality — whether a person can solve the problem themselves, and whether they leave the funnel.
We separated these concepts. A frequent but minor problem can cause momentary irritation. A rare but critical problem can lead to a client leaving for a competitor—therefore, they should be prioritized in the backlog.
Research results: aggregated data
We quantitatively confirmed the client's hypothesis. The suspected problem turned out to be the most massive—it affected more than half of the participants. In total, we found 11 barriers across five stages of the funnel.
However, the highest value to the project came from barriers that were not in the bank's hypotheses. They cannot be detected through product analytics or expert audit:
- Infrastructure failure. Several respondents in a row could not pass the identification step. In the bank's analytics, this looked like a user error. Timecodes showed that the failure was localized in time—a signal of backend instability.
- Error architecture. Difficulties with validation or data access reset the scenario to the very beginning. Minor flaws added up to a systemic pattern. The client needed a change in error handling architecture, not a screen redesign.
- Trust barrier. Some people refused to provide data due to privacy concerns. Improving the interface does not work here. The product needed an alternative path.
- Communication transparency. Respondents did not understand financial terminology. Clients left to compare conditions on external platforms, from which they could leave for competitors.
Impact for the product
The bank received a sprint-ready backlog:
- A prioritized list of 11 barriers with exact frequency.
- 97 documented insights with video confirmations.
- Task division for teams. Some problems were handed over to interface designers, some to backend developers, and some went to the business requirements level.
Main insight: The most massive problem was at the final step of the funnel. The user had already spent maximum effort filling out the application. Eliminating the barrier at this point will give the business the highest conversion return.
Bottom line
Combining an expert approach with specialized services changes the standard of UX research. AI removed the routine load from researchers, allowing them to focus on the architectural connections of the product. The client received reliable data for conversion growth without overpaying for manual analytical labor.
Let's conduct the same research for your product
Rocket Tech conducts quantitative and qualitative product research in its own AI UX laboratory in Almaty: usability testing, in-depth interviews, testing of prototypes and release products, diary studies, A/B experiments.
Learn more about the types of research for your products on the Discovery page.
