Good choices about data paging are an important part of design and development. Sometimes we need to get lists of data from the server, and sometimes these lists can be really long. Breaking lists up into smaller, discreet "pages" can reduce server overhead and improve response time.
Paging choices should start on day one of the design. Designers and developers should be asking, "How large will these lists get? What is a reasonable page size?" The way these lists are treated have implications for performance, engineering cost, hosting cost, and ultimately client satisfaction.
Let’s start by looking at some of the bad things that can happen without paging, or when using incorrect paging:
Potential Issues: Server / API
- Slower APIs: Extra processing time is incurred on the server side, which adds theoretically unbounded response times.
- Added hosting cost: The extra processing time on the server correlates to additional hardware requirements that correlates to additional client hosting costs.
- Wasted maintenance cycles: Time is eventually spent to correct these problems during later optimizations, trying to improve response times without addressing the number of returned records is a fool’s errand.
Potential Issues: Web and Native Clients
- Slower clients: The client suffers doubly, from the slower API and again from increased rendering time and decreased UI performance.
- Unresponsive clients and crashes: In the extreme, rendering lists of thousands of items in a web client can cause the UI to temporarily lock. This problem can be particularly severe in Angular if done without care. Crashes are possible in really bad cases.
- Decreased usability: From a UX perspective, a list can be too long and it’s important that design considers the implication of lists based on how long they may be. A US state list will always be around 50 items, but a list of every city in even a single state is probably too long to scroll through.
Fortunately, all of the above problems have simple, recognized solutions. It’s up to engineers and designers to recognize these potential issues and put good solutions in place up front. The solutions below are a few common patterns; the order doesn’t correspond to the issues listed above.
Potential Solutions: Server / API
- Good page size: 20 is a standard size, but this really depends on what the cost of getting a larger list is as well as what a reasonable list on the client will be. From the perspective that VOKAL is usually building APIs that have a client, the client UI serves as a useful guide here. There is also a big difference between an API endpoint with 4 joins and one that just grabs the top items on an indexed column.
- Filters on the API: Clients often need filtered data, and if there are filter criteria available on the API requests, the database can potentially provide a smaller result set instead of multiple pages of data, or a single large result set. In some cases, filters might even be required for requests.
- Sorting on the API: This addresses a related problem, that requiring individual clients to each implement sorting separately requires additional work, makes paging impractical and tends to lead to different clients having different implementations, which may be considered a bug.
Potential Solutions: Web and Native Clients
- Infinite Scroll: The old standby, exemplified by Pinterest, this is a solution to lists that don’t have a length that can be limited, and don’t require any client-side sorting. API requests are made as the user scrolls down.
- Required filters: If the API requires a filter, there clearly needs to be a filter required on the client, likely something with a default. In addition, filters provide more useful result sets that is better UX than lists containing hundreds of items.
- Better designs: Sometimes large lists are just a design problem that may originate in unclear, or poor requirements, or a lack of understanding from UX and UI designers about what the data will actually be.
Ultimately, because paging touches so many parts of the design, client and server, it’s up to everyone involved in each project to look out for the issues and collaborate on solutions.