background

A new package was built to replace the system for the purpose of business improvement.

The number of tasks that could not be handled by the current system was increasing, and the increase in labor costs due to this was a current issue. To solve this problem, a replacement of the current system was planned.

It was decided to introduce a relatively inexpensive package rather than scratch development. In order to avoid confusion due to changes in operations after the introduction of the system, it was decided to introduce a package specialized for each operation without changing the workflow as much as possible, and to use add-ons for data linkage.

The project went smoothly and was completed within the planned period. However, soon after the introduction of the packages, frequent system failures occurred, and the workload increased more than before due to an increase in user inquiries and operational work. This forced the company to fundamentally resolve the failures and revise the plan.

issue

The introduction of packages to reduce the load has conversely increased the load.

Users complained that they could not use the system for their work, and the system administrators were forced to deal with the increased operational workload.

The survey identified the following five issues based on the incidents that occurred.

[issue] (1)
Multiple packages were used together for one system.

[Occurrence]
Abnormal termination in the data linkage process between packages.
As a result, user input ends normally but data linkage between packages ends abnormally.

[issue] (2)
Add-ons were added to control drastic changes in operations while taking advantage of the package’s features.

[Occurrence]
Some screen operations create operations that generate unnecessary transactions.
If users interrupt screen operations halfway, only the operation before the interruption will result in a transaction, which makes data fraud.

[issue] (3)
Restrictions of the package were described in user manuals, and parallel operation with current system was implemented.

[Occurrence]
The user was manipulated outside of own role and unauthorized data was generated.
Although the package is a flexible check that can accommodate various user inputs, the linked data is strictly created for each role, which caused inconsistencies and led to errors.

[issue] (4)
Key items for identifying data in the upstream system were set to IF items.

[Occurrence]
Data cannot be referenced from upstream systems, even by system operators .
When key items were set up to identify data in the upstream system, the data was held only in a private DB, necessitating a method of text search and investigation from the files.

[issue] (5)
In the event of a failure, the highest priority was given to involving the vendor to deal with the problem and investigate the cause.

[Occurrence]
Failure occurred outside of regular hours and investigation was stopped because the vendor staff could not be reached.
About a month after the system went into production and before switching to a maintenance contract, a system error occurred on the package screen. We were unable to determine whether the problem was with the data linked from the upstream system or with the package or add-ons, so we investigated only by using our imagination based on the information available until we could contact the vendor representative.

activity

Reorganized operations to take advantage of features of the package to be introduced.

From the analysis of the issues, it was inferred that there were problems in the selection and implementation of the package. It was decided to implement initiatives to solve each of these issues.

[issue] (1)
Multiple packages were used together for one system.(Figure 1)

[background]
– Each package has its own specific philosophy, and no two packages have the same philosophy. Due to differences in ideology, the criteria for data checking and realization methods differ, so the range of available data also differs. Therefore, considering the disadvantages caused by differences in ideology, we recommend the introduction of a single package.

– Vendors are not allowed to respond to changes that go against the philosophy of the package. Even if an add-on is used to solve the problem, if the change is contrary to the philosophy of the package, the vendor will refuse to support the change for the sake of quality assurance of the package. Consider issuing reminders in user manuals and system support.

– Unique support between packages is difficult. Since the design and implementation of the packages are not public, it is difficult to add appropriate support for both add-ons by the vendor and implementation by proprietary support, because sufficient review and testing cannot be conducted.

(Figure 1)

[issue] (2)
Add-ons were added to control drastic changes in business operations while taking advantage of the package’s features. (Figure 2)

[background]
– It is difficult to determine the impact of a change in the transaction’s unit of occurrence.
Transactions have multiple paths of origin, and it is difficult to change the unit of occurrence for all of them.

– “just test it to make sure it works” doesn’t work.
Packages tend to be tested only for business patterns under the assumption that they are guaranteed.
As a result, exception patterns cannot be fully tested, which often leads to production failures.

(Figure 2)

[issue](3)
The restrictions of the package were described in the user manual,
and parallel operation with the current system was implemented. (Figure 3)

[background]
– The package’s standard features do not always allow for the authorization settings that a company expects. The package has standard authorization settings and may not be able to meet the company’s specific authorization settings.

– The user somehow manages to do the work on own, so they make an input that is not expected by the system. Users perform operations that the system does not expect because of a lack of understanding or insufficient description in the manuals or to deal with exceptions. Standard features of the package may not be able to limit input.

(Figure 3)

[issue] (4)
Key items for identifying data in the upstream system were set to IF items. (Figure 4)

[background]
– Not only the linkage availability, but also the search method for linked data and its operability should be kept in mind for maintenance and operation.

– Log output granularity, IF file storage method, reference range and searchability. Since the information displayed on the screen is part of the retained data, it is important to determine the extent to which users and system personnel can refer to data that cannot be confirmed on the screen.

(Figure 4)

[issue] (5)
When a failure occurred, the vendor was involved, and the highest priority was given to dealing with the problem and investigating the cause.(Figure 5)

[background]
– Vendors who can see the logic of packages are essential to isolate the cause of failure. Failures can be caused by master configuration errors, lack of user compliance with rules, system operation errors, etc. Since the internal structure of the package is not open to the public, vendor cooperation is essential from the investigation onward.

– Package vendors do not hold down the business.
Vendors only have the necessary business knowledge to provide the package and do not hold the details.
Care should be taken not to confuse familiarity with the package with familiarity with the business when considering operation and maintenance after the package is introduced.

(Figure 5)

consequence

By preventing common problems with package implementation, we were able to smoothly transition and achieve a significant improvement in operations.
By establishing operation rules and mechanisms and minimizing manual operations, the company was able to reduce the number of errors caused by user operation errors.

In addition, as a result of the time spent on user education, the number of failures and inquiries after the production launch was reduced to a level lower than expected.