Recherche avancée

Sur d’autres sites (5572)

• Data Privacy Regulations : Essential Knowledge for Global Business

  6 mars, par Daniel Crough

  If you run a website that collects visitors’ data, you might be violating privacy regulations somewhere in the world. At last count, over 160 countries have privacy laws — and your customers in those countries know about them.

  A recent survey found that 53% of people who answered know about privacy rules in their country and want to follow them. This is up from 46% two years ago. Furthermore, customers increasingly want to buy from businesses they can trust with their data.

  That’s why businesses must take data privacy seriously. In this article, we’ll first examine data privacy rules, why we need them, and how they are enforced worldwide. Finally, we’ll explore strategies to ensure compliance and tools that can help.

  What are data privacy regulations ?

  Let’s first consider data privacy. What is it ? The short answer is individuals’ ability to control their personal information. That’s why we need laws and rules to let people decide how their data is collected, used, and shared. Crucially, the laws empower individuals to withdraw permission to use their data anytime.

  The UNCTAD reports that only 13 countries had data protection laws or rules before the 2000s. Many existed before businesses could offer online services, so they needed updating. Today, 162 national laws protect data privacy, half of which emerged in the last decade.

  Why is this regulation necessary ?

  There are many reasons, but the impetus comes from consumers who want their governments to protect their data from exploitation. They understand that participating in the digital economy means sharing personal information like email addresses and telephone numbers, but they want to minimise the risks of doing so.

  Data privacy regulation is essential for :

  • Protecting personal information from exploitation with transparent rules and guidelines on handling it securely.
  • Implementing adequate security measures to prevent data breaches.
  • Enforcing accountability for how data is collected, stored and processed.
  • Giving consumers control over their data.
  • Controlling the flow of data across international borders in a way that fully complies with the regulations.
  • Penalising companies that violate privacy laws.

  Isn’t it just needless red tape ?

  Data breaches in recent years have been one of the biggest instigators of the increase in data privacy regulations. A list of the top ten data breaches illustrates the point.

  #	Company	Location	Year	# of Records	Data Type
  1	Yahoo	Global	2013	3B	user account information
  2	Aadhaar	India	2018	1.1B	citizens’ ID/biometric data
  2	Alibaba	China	2019	1.1B	users’ personal data
  4	LinkedIn	Global	2021	700M	users’ personal data
  5	Sina Weibo	China	2020	538M	users’ personal data
  6	Facebook	Global	2019	533M	users’ personal data
  7	Marriott Int’l	Global	2018	500M	customers’ personal data
  8	Yahoo	Global	2014	500M	user account information
  9	Adult Friend Finder	Global	2016	412.2M	user account information
  10	MySpace	USA	2013	360M	user account information

  And that’s just the tip of the iceberg. Between November 2005 and November 2015, the US-based Identity Theft Resource Center counted 5,754 data breaches that exposed 856,548,312 records, mainly in that country.

  It’s no wonder that citizens worldwide want organisations they share their personal data with to protect that data as if it were their own. More specifically, they want their governments to :

  • Protect their consumer rights
  • Prevent identity theft and other consumer fraud
  • Build trust between consumers and businesses
  • Improve cybersecurity measures
  • Promote ethical business practices
  • Uphold international standards

  Organisations using personal data in their operations want to minimise financial and reputational risk. That’s common sense, especially when external attacks cause 68% of data breaches.

  The terminology of data privacy

  With 162 national laws already in place, the legal space surrounding data privacy grows more complex every day. Michalsons has a list of different privacy laws and regulations in force in significant markets around the world.

  Fortunately, there’s plenty of commonality for two reasons : first, all countries want to solve the same problem ; second, those drafting the legislation have adopted much of what other countries have already developed. As a result, the terminology remains almost the same, even when the language changes.

  These are the core concepts at play :

  Term	Definition
  Access and control	Consumers can access, review, edit and delete their data
  Data protection	Organisations must protect data from being stolen or compromised
  Consumer consent	Consumers can grant and withdraw or refuse access to their data
  Deletion	Consumers can request to have their data erased
  Data breach	When the security of data has been compromised
  Data governance	The management of data within an organisation
  Double opt-in	Two-factor authentication to add a layer of confirmation
  GDPR	Governing data privacy in Europe since 2016
  Personally identifiable information (PII)	Data used to identify, locate, or contact an individual
  Pseudonymisation	Replace personal identifiers with artificial identifiers or pseudonyms
  Publicly available information	Data from official sources, without restrictions on access or use
  Rectification	Consumers can request to have errors in their data corrected

  Overview of current data privacy legislation

  Over three-quarters of the world has formulated and rolled out data privacy legislation — or is currently doing so. Here’s a breakdown of the laws and regulations you can expect to find in most significant markets worldwide.

  Europe

  Thoughts of protecting data privacy first occurred in Europe when the German government became concerned about automated data processing in 1970. A few years later, Sweden was the first country to enact a law requiring permits for processing personal data, establishing the first data protection authority.

  General Data Protection Regulation (GDPR)

  Sweden’s efforts triggered a succession of European laws and regulations that culminated in the European Union (EU) GDPR, enacted in 2016 and enforced from 25 May 2018. It’s a detailed and comprehensive privacy law that safeguards the personal data and privacy of EU citizens.

  The main objectives of GDPR are :

  • Strengthening the privacy rights of individuals by empowering them to control their data.
  • Establishing a uniform data framework for data privacy across the EU.
  • Improving transparency and accountability by mandating businesses to handle personal data responsibly and fully disclose how they use it.
  • Extending the regulation’s reach to organisations external to the EU that collect, store and process the data of EU residents.
  • Requiring organisations to conduct Protection Impact Assessments (PIAs) for “high-risk” projects.

  ePrivacy Regulation on Privacy and Electronic Communications (PECR)

  The second pillar of the EU’s strategy to regulate the personal data of its citizens is the ePrivacy Regulation on Privacy and Electronic Communications (EU PECR). Together with the GDPR, it will comprise data protection law in the union. This regulation applies to :

  • Providers of messaging services like WhatsApp, Facebook and Skype
  • Website owners
  • Owners of apps that have electronic communication components
  • Commercial direct marketers
  • Political parties sending promotional messages electronically
  • Telecommunications companies
  • ISPs and WiFi connection providers

  The EU PECR was intended to commence with GDPR on 25 May 2018. That didn’t happen, and as of January 2025, it was in the process of being redrafted.

  EU Data Act

  One class of data isn’t covered by GDPR or PECR : internet product-generated data. The EU Data Act provides the regulatory framework to govern this data, and it applies to manufacturers, suppliers, and users of IoT devices or related services.

  The intention is to facilitate data sharing, use, and reuse and to facilitate organisations’ switching to a different cloud service provider. The EU Data Act entered into force on 11 January 2024 and is applicable from September 2025.

  GDPR UK

  Before Brexit, the EU GDPR was in force in the UK. After Brexit in 2020, the UK opted to retain the regulations as UK GDPR but asserted independence to keep the framework under review. It’s part of a wider package of reform to the data protection environment that includes the Data Protection Act 2018 and the UK PECR.

  In the USA

  The primary federal law regarding data privacy in the US is the Privacy Act of 1974, which has been in revision for some time. However, rather than wait for the outcome of that process, many business sectors and states have implemented their own measures.

  Sector-specific data protection laws

  This sectoral approach to data protection relies on a combination of legislation, regulation and self-regulation rather than governmental control. Since the mid-1990s, the country has allowed the private sector to lead on data protection, resulting in ad hoc legislation arising when circumstances require it. Examples include the Video Privacy Protection Act of 1988, the Cable Television Protection and Competition Act of 1992 and the Fair Credit Reporting Act.

  

  California Consumer Privacy Act (CCPA)

  California was the first state to act when federal privacy law development stalled. In 2018, it enacted the California Consumer Privacy Act (CCPA) to protect and enforce Californians’ rights regarding the privacy of their personal information. It came into force in 2020.

  California Privacy Act (CPRA)

  In November of that same year, California voters approved the California Privacy Rights Act (CPRA). Billed as the strongest consumer privacy law ever enacted in the US, CPRA works with CCPA and adds the best elements of laws and regulations in other jurisdictions (Europe, Japan, Israel, New Zealand, Canada, etc.) into California’s personal data protection regime.

  Virginia Consumer Data Protection Act (CDPA)

  In March 2021, Virginia became the next US state to implement privacy legislation. The Virginia Consumer Data Protection Act (VCDPA), which is also informed by global legislative developments, tries to strike a balance between consumer privacy protections and business interests. It governs how businesses collect, use, and share consumer data.

  Colorado Privacy Act (CPA)

  Developed around the same time as VCDPA, the Colorado Privacy Act (CPA) was informed by that law and GDPR and CCPA. Signed into law in July 2021, the CPA gives Colorado residents more control over their data and establishes guidelines for businesses on handling the data.

  Other states generally

  Soon after, additional states followed suit and, similar to Colorado, examined existing legislation to inform the development of their own data privacy laws and regulations. At the time of writing, the states with data privacy laws at various stages of development were Connecticut, Florida, Indiana, Iowa, Montana, New York, Oregon, Tennessee, Texas, and Utah.

  By the time you read this article, more states may be doing it, and the efforts of some may have led to laws and regulations coming into force. If you’re already doing business or planning to do business in the US, you should do your own research on the home states of your customers.

  Globally

  Beyond Europe and the US, other countries are also implementing privacy regulations. Some were well ahead of the trend. For example, Chile’s Law on the Protection of Private Life was put on the books in 1999, while Mauritius enacted its first Data Protection Act in 2004 — a second one came along in 2017 to replace it.

  Canada

  The regulatory landscape around data privacy in Canada is as complicated as it is in the US. At a federal government level, there are two laws : The Privacy Act for public sector institutions and the Personal Information Protection and Electronic Documents Act (PIPEDA) for the private sector.

  PIPEDA is the one to consider here. Like all other data privacy policies, it provides a framework for organisations handling consumers’ personal data in Canada. Although not quite up to GDPR standard, there are moves afoot to close that gap.

  The Digital Charter Implementation Act, 2022 (aka Bill C-27) is proposed legislation introduced by federal agencies in June 2022. It’s intended to align Canada’s privacy framework with global standards, such as GDPR, and address emerging digital economy challenges. It may or may not have been finalised when you read this.

  At the provincial level, three of Canada’s provinces—Alberta, British Columbia, and Quebec—have introduced laws and regulations of their own. Their rationale was similar to that of Bill C-27, so they may become redundant if and when that bill passes.

  Japan

  Until recently, Japan’s Act on the Protection of Personal Information (APPI) was considered by many to be the most comprehensive data protection law in Asia. Initially introduced in 2003, it was significantly amended in 2020 to align with global privacy standards, such as GDPR.

  APPI sets out unambiguous rules for how businesses and organisations collect, use, and protect personal information. It also sets conditions for transferring the personal information of Japanese residents outside of Japan.

  

  China

  The new, at least for now, most comprehensive data privacy law in Asia is China’s Personal Information Protection Law (PIPL). It’s part of the country’s rapidly evolving data governance framework, alongside the Cybersecurity Law and the Data Security Law.

  PIPL came into effect in November 2021 and was informed by GDPR and Japan’s APPI, among others. The data protection regime establishes a framework for protecting personal information and imposes significant compliance obligations on businesses operating in China or targeting consumers in that country.

  Other countries

  Many other nations have already brought in legislation and regulations or are in the process of developing them. As mentioned earlier, there are 162 of them at this point, and they include :

  Argentina	Costa Rica	Paraguay
  Australia	Ecuador	Peru
  Bahrain	Hong Kong	Saudi Arabia
  Bermuda	Israel	Singapore
  Brazil	Mauritius	South Africa
  Chile	Mexico	UAE
  Colombia	New Zealand	Uruguay

  Observant readers might have noticed that only two countries in Africa are on that list. More than half of the 55 countries on the continent have or are working on data privacy legislation.

  It’s a complex landscape

  Building a globalised business model has become very complicated, with so much legislation already in play and more coming. What you must do depends on the countries you plan to operate in or target. And that’s before you consider the agreements groups of countries have entered into to ease the flow of personal data between them.

  In this regard, the EU-US relationship is instructive. When GDPR came into force in 2016, so did the EU-US Privacy Shield. However, about four years later, the Court of Justice of the European Union (CJEU) invalidated it. The court ruled that the Privacy Shield didn’t adequately protect personal data transferred from the EU to the US.

  The ruling was based on US laws that allow excessive government surveillance of personal data transferred to the US. The CJEU found that this conflicted with the basic rights of EU citizens under the European Union’s Charter of Fundamental Rights.

  A replacement was negotiated in a new mechanism : the EU-US Data Privacy Framework. However, legal challenges are expected, and its long-term viability is uncertain. The APEC Privacy Framework and the OECD Privacy Framework, both involving the US, also exist.

  

  Penalties for non-compliance

  Whichever way you look at it, consumer data privacy laws and regulations make sense. But what’s really interesting is that many of them have real teeth to punish offenders. GDPR is a great example. It was largely an EU concern until January 2022 when the French data protection regulator hit Google and Facebook with serious fines and criminal penalties.

  Google was fined €150M, and Facebook was told to pay €60M for failing to allow French users to reject cookie tracking technology easily. That started a tsunami of ever-larger fines.

  The largest so far was the €1.2B fine levied by the Irish Data Protection Commission on Meta, the owner of Instagram, Facebook, and WhatsApp. It was issued for transferring European users’ personal data to the US without adequate data protection mechanisms. This significant penalty demonstrated the serious financial implications of non-compliance.

  These penalties follow a structured approach rather than arbitrary determinations. The GDPR defines an unambiguous framework for fines. They can be up to 4% of a company’s total global turnover in the previous fiscal year. That’s a serious business threat.

  What should you do ?

  For businesses committed to long-term success, accepting and adapting to regulatory requirements is essential. Data privacy regulations and protection impact assessments are here to stay, with many national governments implementing similar frameworks.

  However, there is some good news. As you’ve seen, many of these laws and regulations were informed by GDPR or retrospectively aligned. That’s a good place to start. Choose tools to handle your customer’s data that are natively GDPR-compliant.

  For example, web analytics is all about data, and a lot of that data is personal. And if, like many people, you use Google Analytics 4, you’re already in trouble because it’s not GDPR-compliant by default. And achieving compliance requires significant additional configuration.

  A better option would be to choose a web analytics platform that is compliant with GDPR right off the bat. Something like Matomo would do the trick. Then, complying with any of the tweaks individual countries have made to the basic GDPR framework will be a lot easier—and may even be handled for you.

  Privacy-centric data strategies

  Effective website data analysis is essential for business success. It enables organisations to understand customer needs and improve service delivery.

  But that data doesn’t necessarily need to be tied to their identity — and that’s at the root of many of these regulations.

  It’s not to stop companies from collecting data but to encourage and enforce responsible and ethical handling of that data. Without an official privacy policy or ethical data collection practices, the temptation for some to use and abuse that data for financial gain seems too great to resist.

  

  Cookie usage and compliance

  There was a time when cookies were the only way to collect reliable information about your customers and prospects. But under GDPR, and in many countries that based or aligned their laws with GDPR, businesses have to give users an easy way to opt out of all tracking, particularly tracking cookies.

  So, how do you collect the information you need without cookies ? Easy. You use a web analytics platform that doesn’t depend wholly on cookies. For example, in certain countries and when configured for maximum privacy, Matomo allows for cookieless operation. It can also help you manage the cookie consent requirements of various data privacy regulations.

  Choose the right tools

  Data privacy regulations have become a permanent feature of the global business landscape. As digital commerce continues to expand, these regulatory frameworks will only become more established. Fortunately, there is a practical approach forward.

  As mentioned several times, GDPR is considered by many countries to be a particularly good example of effective data privacy regulation. For that reason, many of them model their own legislation on the EU’s effort, making a few tweaks here and there to satisfy local requirements or anomalies.

  As a result, if you comply with GDPR, the chances are that you’ll also comply with many of the other data privacy regulations discussed here. That also means that you can select tools for your data harvesting and analytics that comply with the GDPR out of the box, so to speak. Tools like Matomo.

  Matomo lets website visitors retain full control over their data.

  Before deciding whether to go with Matomo On-premise or the EU-hosted cloud version, why not start your 21-day free trial ? No credit card required.

• A Quick Start Guide to the Payment Services Directive (PSD2)

  22 novembre 2024, par Daniel Crough — Banking and Financial Services, Privacy

  In 2023, there were 266.2 billion real-time payments indicating that the demand for secure transactions has never been higher. As we move towards a more open banking system, there are a host of new payment solutions that offer convenience and efficiency, but they also present new risks.

  The Payment Services Directive 2 (PSD2) is one of many regulations established to address these concerns. PSD2 is a European Union (EU) business initiative to offer smooth payment experiences while helping customers feel safe from online threats. 

  In this post, learn what PSD2 includes, how it improves security for online payments, and how Matomo supports banks and financial institutions with PSD2 compliance.

  What is PSD2 ? 

  PSD2 is an EU directive that aims to improve the security of electronic payments across the EU. It enforces strong customer authentication and allows third-party access to consumer accounts with explicit consent. 

  Its main objectives are :

  • Strengthening security and data privacy measures around digital payments.
  • Encouraging innovation by allowing third-party providers access to banking data.
  • Improving transparency with clear communication regarding fees, terms and conditions associated with payment services.
  • Establishing a framework for sharing customer data securely through APIs for PSD2 open banking.

  Rationale behind PSD2 

  PSD2’s primary purpose is to engineer a more integrated and efficient European payment market without compromising the security of online transactions. 

  The original directive aimed to standardise payment services across EU member states, but as technology evolved, an updated version was needed.

  PSD2 is mandatory for various entities within the European Economic Area (EEA), like :

  • Banks and credit institutions
  • Electronic money institutions or digital banks like Revolut
  • Card issuing and acquiring institutions
  • Fintech companies
  • Multi-national organisations operating in the EU

  PSD2 implementation timeline

  With several important milestones, PSD2 has reshaped how payment services work in Europe. Here’s a closer look at the pivotal events that paved the way for its launch.

  • 2002 : The banking industry creates the European Payments Council (EC), which drives the Single Euro Payments Area (SEPA) initiative to include non-cash payment instruments across European regions. 
  • 2007 : PSD1 goes into effect.
  • 2013 : EC proposes PSD2 to include protocols for upcoming payment services.
  • 2015 : The Council of European Union passes PSD2 and gives member states two years to incorporate it.
  • 2018 : PSD2 goes into effect. 
  • 2019 : The final deadline for all companies within the EU to comply with PSD2’s regulations and rules for strong customer authentication. 

  PSD2 : Key components 

  PSD2 introduces several key components. Let’s take a look at each one.

  Strong Customer Authentication (SCA)

  The Regulatory Technical Standards (RTS) under PSD2 outline specific requirements for SCA. 

  SCA requires multi-factor authentication for online transactions. When customers make a payment online, they need to verify their identity using at least two of the three following elements :

  • Knowledge : Something they know (like a password, a code or a secret answer)
  • Possession : Something they have (like their phone or card)
  • Inherence : Something they are (like biometrics — fingerprints or facial features)

  

  Before SCA, banks verified an individual’s identity only using a password. This dual verification allows only authorised users to complete transactions. SCA implementation reduces fraud and increases the security of electronic payments.

  SCA implementation varies for different payment methods. Debit and credit cards use the 3D Secure (3DS) protocol. E-wallets and other local payment measures often have their own SCA-compliant steps. 

  3DS is an extra step to authenticate a customer’s identity. Most European debit and credit card companies implement it. Also, in case of fraudulent chargebacks, the issuing bank becomes liable due to 3DS, not the business. 

  However, in SCA, certain transactions are exempt : 

  • Low-risk transactions : A transaction by an issuer or an acquirer whose fraud level is below a specific threshold. If the acquirer feels that a transaction is low risk, they can request to skip SCA. 
  • Low-value transactions : Transactions under €30.
  • Trusted beneficiaries : Trusted merchants customers choose to safelist.
  • Recurring payments : Recurring transactions for a fixed amount are exempt from SCA after the first transaction.

  Third-party payment service providers (TPPs) framework

  TPPs are entities authorised to access customer banking data and initiate payments. There are three types of TPPs :

  Account Information Service Providers (AISPs)

  AISPs are services that can view customers’ account details, but only with their permission. For example, a budgeting app might use AISP services to gather transaction data from a user’s bank account, helping them monitor expenses and oversee finances. 

  Payment Initiation Service Providers (PISPs)

  PISPs enable clients to initiate payments directly from their bank accounts, bypassing the need for conventional payment options such as debit or credit cards. After the customer makes a payment, PISPs immediately contact the merchant to ensure the user can access the online services or products they bought. 

  Card-Based Payment Instruments (CBPII)

  CBPIIs refer to services that issue payment cards linked to customer accounts. 

  Requirements for TPPs

  To operate effectively under PSD2, TPPs must meet several requirements :

  Consumer consent : Customers must explicitly authorise TPPs to retrieve their financial data. This way, users can control who can view their information and for what purpose.

  Security compliance : TPPs must follow SCA and secure communication guidelines to protect users from fraud and unauthorised access.

  API availability : Banks must make their Application Programming Interfaces (APIs) accessible and allow TPPs to connect securely with the bank’s systems. This availability helps in easy integration and lets TPPs access essential data. 

  Consumer protection methods

  PSD2 implements various consumer protection measures to increase trust and transparency between consumers and financial institutions. Here’s a closer look at some of these key methods :

  • Prohibition of unjustified fees : PSD2 requires banks to clearly communicate any additional charges or fees for international transfers or account maintenance. This ensures consumers are fully aware of the actual costs and charges.
  • Timely complaint resolution : PSD2 mandates that payment service providers (PSPs) have a straightforward complaint procedure. If a customer faces any problems, the provider must respond within 15 business days. This requirement encourages consumers to engage more confidently with financial services.
  • Refund in case of unauthorised payment : Customers are entitled to a full refund for payments made without their consent.
  • Surcharge ban : Additional charges on credit and debit card payments aren’t allowed. Businesses can’t impose extra fees on these payment methods, which increases customers’ purchasing power.

  Benefits of PSD2 

  Businesses — particularly those in banking, fintech, finserv, etc. — stand to benefit from PSD2 in several ways.

  Access to customer data

  With customer consent, banks can analyse spending patterns to develop tailored financial products that match customer needs, from personalised savings accounts to more relevant loan offerings.

  Innovation and cost benefits 

  PSD2 opened payment processing up to more market competition. New payment companies bring fresh approaches to banking services, making daily transactions more efficient while driving down processing fees across the sector.

  Also, banks now work alongside payment technology providers, combining their strengths to create better services. This collaboration brings faster payment options to businesses, helping them stay competitive while reducing operational costs.

  Improved customer trust and experience

  Due to PSD2 guidelines, modern systems handle transactions quickly without compromising the safety of payment data, creating a balanced approach to digital banking.

  

  Banking customers now have more control over their financial information. Clear processes allow consumers to view and adjust their financial preferences as needed.

  Strong security standards form the foundation of these new payment systems. Payment provider platforms must adhere to strict regulations and implement additional protection measures.

  Challenges in PSD2 compliance 

  What challenges can banks and financial institutions face regarding PSD2 compliance ? Let’s examine them. 

  Resource requirements

  For many businesses, the new requirements come with a high price tag. PSD2 requires banks and fintechs to build and update their systems so that other providers can access customer data safely. For example, they must develop APIs to allow TPPs to acquire customer data. 

  Many banks still use older systems that can’t meet PSD2’s added requirements. In addition to the cost of upgrades, complying with PSD2 requires banks to devote resources to training staff and monitoring compliance.

  The significant costs required to update legacy systems and IT infrastructure while keeping services running remain challenging.

  Risks and penalties

  Organisations that fail to comply with PSD2 regulations can face significant penalties.

  Additionally, the overlapping requirements of PSD2 and other regulations, such as the General Data Protection Regulation (GDPR), can create confusion. 

  Banks need clear agreements with TPPs about who’s responsible when things go wrong. This includes handling data breaches, preventing data misuse and protecting customer information. 

  Increased competition 

  Introducing new players in the financial ecosystem, such as AISPs and PISPs, creates competition. Banks must adapt their services to stay competitive while managing compliance costs.

  PSD2 aims to protect customers but the stronger authentication requirements can make banking less convenient. Banks must balance security with user experience. Focused time, effort and continuous monitoring are needed for businesses to stay compliant and competitive.

  How Matomo can help 

  Matomo gives banks and financial institutions complete control over their data through privacy-focused web analytics, keeping collected information internal rather than being used for marketing or other purposes. 

  Its advanced security setup includes access controls, audit logs, SSL encryption, single sign-on and two-factor authentication. This creates a secure environment where sensitive data remains accessible only to authorised staff.

  While prioritizing privacy, Matomo provides tools to understand user flow and customer segments, such as session recordings, heatmaps and A/B testing.

  Financial institutions particularly benefit from several key features : 

  • Tools for obtaining explicit consent before processing personal data like this Do Not Track preference
  • Insights into how financial institutions integrate TPPs (including API usage, user engagement and potential authentication drop-off points)
  • Tracking of failed login attempts or unusual access patterns
  • IP anonymization to analyse traffic patterns and detect potential fraud

  

  PSD3 : The next step 

  In recent years, we have seen the rise of innovative payment companies and increasingly clever fraud schemes. This has prompted regulators to propose updates to payment rules.

  PSD3’s scope is to adapt to the evolving digital transformation and to better handle these fraud risks. The proposed measures : 

  • Encourage PSPs to share fraud-related information.
  • Make customers aware of the different types of fraud.
  • Strengthen customer authentication standards.
  • Provide non-bank PSPs restricted access to EU payment systems. 
  • Enact payment rules in a directly applicable regulation and harmonise and enforce the directive.

  Web analytics that respect user privacy 

  Achieving compliance with PSD2 may be a long road for some businesses. With Matomo, organisations can enjoy peace of mind knowing their data practices align with legal requirements.

  Ready to stop worrying over compliance with regulations like PSD2 and take control of your data ? Start your 21-day free trial with Matomo.

• Developing MobyCAIRO

  26 mai 2021, par Multimedia Mike — General

  I recently published a tool called MobyCAIRO. The ‘CAIRO’ part stands for Computer-Assisted Image ROtation, while the ‘Moby’ prefix refers to its role in helping process artifact image scans to submit to the MobyGames database. The tool is meant to provide an accelerated workflow for rotating and cropping image scans. It works on both Windows and Linux. Hopefully, it can solve similar workflow problems for other people.

  As of this writing, MobyCAIRO has not been tested on Mac OS X yet– I expect some issues there that should be easily solvable if someone cares to test it.

  The rest of this post describes my motivations and how I arrived at the solution.

  Background
  I have scanned well in excess of 2100 images for MobyGames and other purposes in the past 16 years or so. The workflow looks like this :

  
  

  

  Image workflow

  
  

  It should be noted that my original workflow featured me manually rotating the artifact on the scanner bed in order to ensure straightness, because I guess I thought that rotate functions in image editing programs constituted dark, unholy magic or something. So my workflow used to be even more arduous :

  
  

  

  I can’t believe I had the patience to do this for hundreds of scans

  
  

  Sometime last year, I was sitting down to perform some more scanning and found myself dreading the oncoming tedium of straightening and cropping the images. This prompted a pivotal question :

  
  Why can’t a computer do this for me ?
  

  After all, I have always been a huge proponent of making computers handle the most tedious, repetitive, mind-numbing, and error-prone tasks. So I did some web searching to find if there were any solutions that dealt with this. I also consulted with some like-minded folks who have to cope with the same tedious workflow.

  I came up empty-handed. So I endeavored to develop my own solution.

  Problem Statement and Prior Work
  
  I want to develop a workflow that can automatically rotate an image so that it is straight, and also find the most likely crop rectangle, uniformly whitening the area outside of the crop area (in the case of circles).

  As mentioned, I checked to see if any other programs can handle this, starting with my usual workhorse, Photoshop Elements. But I can’t expect the trimmed down version to do everything. I tried to find out if its big brother could handle the task, but couldn’t find a definitive answer on that. Nor could I find any other tools that seem to take an interest in optimizing this particular workflow.

  When I brought this up to some peers, I received some suggestions, including an idea that the venerable GIMP had a feature like this, but I could not find any evidence. Further, I would get responses of “Program XYZ can do image rotation and cropping.” I had to tamp down on the snark to avoid saying “Wow ! An image editor that can perform rotation AND cropping ? What a game-changer !” Rotation and cropping features are table stakes for any halfway competent image editor for the last 25 or so years at least. I am hoping to find or create a program which can lend a bit of programmatic assistance to the task.

  Why can’t other programs handle this ? The answer seems fairly obvious : Image editing tools are general tools and I want a highly customized workflow. It’s not reasonable to expect a turnkey solution to do this.

  Brainstorming An Approach
  I started with the happiest of happy cases— A disc that needed archiving (a marketing/press assets CD-ROM from a video game company, contents described here) which appeared to have some pretty clear straight lines :

  
  
  

  My idea was to try to find straight lines in the image and then rotate the image so that the image is parallel to the horizontal based on the longest single straight line detected.

  I just needed to figure out how to find a straight line inside of an image. Fortunately, I quickly learned that this is very much a solved problem thanks to something called the Hough transform. As a bonus, I read that this is also the tool I would want to use for finding circles, when I got to that part. The nice thing about knowing the formal algorithm to use is being able to find efficient, optimized libraries which already implement it.

  Early Prototype
  A little searching for how to perform a Hough transform in Python led me first to scikit. I was able to rapidly produce a prototype that did some basic image processing. However, running the Hough transform directly on the image and rotating according to the longest line segment discovered turned out not to yield expected results.

  
  
  

  It also took a very long time to chew on the 3300×3300 raw image– certainly longer than I care to wait for an accelerated workflow concept. The key, however, is that you are apparently not supposed to run the Hough transform on a raw image– you need to compute the edges first, and then attempt to determine which edges are ‘straight’. The recommended algorithm for this step is the Canny edge detector. After applying this, I get the expected rotation :

  
  
  

  The algorithm also completes in a few seconds. So this is a good early result and I was feeling pretty confident. But, again– happiest of happy cases. I should also mention at this point that I had originally envisioned a tool that I would simply run against a scanned image and it would automatically/magically make the image straight, followed by a perfect crop.

  Along came my MobyGames comrade Foxhack to disabuse me of the hope of ever developing a fully automated tool. Just try and find a usefully long straight line in this :

  
  

  Darn it, Foxhack…
  

  There are straight edges, to be sure. But my initial brainstorm of rotating according to the longest straight edge looks infeasible. Further, it’s at this point that we start brainstorming that perhaps we could match on ratings badges such as the standard ESRB badges omnipresent on U.S. video games. This gets into feature detection and complicates things.

  This Needs To Be Interactive
  At this point in the effort, I came to terms with the fact that the solution will need to have some element of interactivity. I will also need to get out of my safe Linux haven and figure out how to develop this on a Windows desktop, something I am not experienced with.

  I initially dreamed up an impressive beast of a program written in C++ that leverages Windows desktop GUI frameworks, OpenGL for display and real-time rotation, GPU acceleration for image analysis and processing tricks, and some novel input concepts. I thought GPU acceleration would be crucial since I have a fairly good GPU on my main Windows desktop and I hear that these things are pretty good at image processing.

  I created a list of prototyping tasks on a Trello board and made a decent amount of headway on prototyping all the various pieces that I would need to tie together in order to make this a reality. But it was ultimately slowgoing when you can only grab an hour or 2 here and there to try to get anything done.

  Settling On A Solution
  Recently, I was determined to get a set of old shareware discs archived. I ripped the data a year ago but I was blocked on the scanning task because I knew that would also involve tedious straightening and cropping. So I finally got all the scans done, which was reasonably quick. But I was determined to not manually post-process them.

  This was fairly recent, but I can’t quite recall how I managed to come across the OpenCV library and its Python bindings. OpenCV is an amazing library that provides a significant toolbox for performing image processing tasks. Not only that, it provides “just enough” UI primitives to be able to quickly create a basic GUI for your program, including image display via multiple windows, buttons, and keyboard/mouse input. Furthermore, OpenCV seems to be plenty fast enough to do everything I need in real time, just with (accelerated where appropriate) CPU processing.

  So I went to work porting the ideas from the simple standalone Python/scikit tool. I thought of a refinement to the straight line detector– instead of just finding the longest straight edge, it creates a histogram of 360 rotation angles, and builds a list of lines corresponding to each angle. Then it sorts the angles by cumulative line length and allows the user to iterate through this list, which will hopefully provide the most likely straightened angle up front. Further, the tool allows making fine adjustments by 1/10 of an angle via the keyboard, not the mouse. It does all this while highlighting in red the straight line segments that are parallel to the horizontal axis, per the current candidate angle.

  
  
  

  The tool draws a light-colored grid over the frame to aid the user in visually verifying the straightness of the image. Further, the program has a mode that allows the user to see the algorithm’s detected edges :

  
  
  

  For the cropping phase, the program uses the Hough circle transform in a similar manner, finding the most likely circles (if the image to be processed is supposed to be a circle) and allowing the user to cycle among them while making precise adjustments via the keyboard, again, rather than the mouse.

  
  
  

  Running the Hough circle transform is a significantly more intensive operation than the line transform. When I ran it on a full 3300×3300 image, it ran for a long time. I didn’t let it run longer than a minute before forcibly ending the program. Is this approach unworkable ? Not quite– It turns out that the transform is just as effective when shrinking the image to 400×400, and completes in under 2 seconds on my Core i5 CPU.

  For rectangular cropping, I just settled on using OpenCV’s built-in region-of-interest (ROI) facility. I tried to intelligently find the best candidate rectangle and allow fine adjustments via the keyboard, but I wasn’t having much success, so I took a path of lesser resistance.

  Packaging and Residual Weirdness
  I realized that this tool would be more useful to a broader Windows-using base of digital preservationists if they didn’t have to install Python, establish a virtual environment, and install the prerequisite dependencies. Thus, I made the effort to figure out how to wrap the entire thing up into a monolithic Windows EXE binary. It is available from the project’s Github release page (another thing I figured out for the sake of this project !).

  The binary is pretty heavy, weighing in at a bit over 50 megabytes. You might advise using compression– it IS compressed ! Before I figured out the --onefile command for pyinstaller.exe, the generated dist/ subdirectory was 150 MB. Among other things, there’s a 30 MB FORTRAN BLAS library packaged in !

  Conclusion and Future Directions
  Once I got it all working with a simple tkinter UI up front in order to select between circle and rectangle crop modes, I unleashed the tool on 60 or so scans in bulk, using the Windows forfiles command (another learning experience). I didn’t put a clock on the effort, but it felt faster. Of course, I was livid with proudness the whole time because I was using my own tool. I just wish I had thought of it sooner. But, really, with 2100+ scans under my belt, I’m just getting started– I literally have thousands more artifacts to scan for preservation.

  The tool isn’t perfect, of course. Just tonight, I threw another scan at MobyCAIRO. Just go ahead and try to find straight lines in this specimen :

  
  
  

  I eventually had to use the text left and right of center to line up against the grid with the manual keyboard adjustments. Still, I’m impressed by how these computer vision algorithms can see patterns I can’t, highlighting lines I never would have guessed at.

  I’m eager to play with OpenCV some more, particularly the video processing functions, perhaps even some GPU-accelerated versions.

  The post Developing MobyCAIRO first appeared on Breaking Eggs And Making Omelettes.