Commit Graph

2536 Commits

Author SHA1 Message Date
Emma Haruka Iwao
3e7b5963e8 Fix a bug where win32::Utf8ToWide and WideToUtf allocated an extra character when converting strings.
Both MultiByteToWideChar and WideCharToMultiByte don't null-terminate the output when the input length is explicitly specified.
See the remarks section of the reference:
https://learn.microsoft.com/en-us/windows/win32/api/stringapiset/nf-stringapiset-widechartomultibyte

PiperOrigin-RevId: 513736037
2023-03-03 15:55:58 +09:00
Toshiyuki Hanaoka
664d0c51ac Add single kanji prediction aggregator
PiperOrigin-RevId: 513714725
2023-03-03 13:20:21 +09:00
Toshiyuki Hanaoka
30cab1275b - Refactor the condition for predictive penalty.
- Update the comment.

Background:
1. The condition was added for enrich partial feature (cl/449889011)
2. Enrich partial was launched and removed the condition for enrich partial (cl/496594650)
  - Here, the check for TYPING_CORRECTION should be removed.

#codehealth

PiperOrigin-RevId: 513677045
2023-03-03 09:54:34 +09:00
Emma Haruka Iwao
968f857b69 Refactor Table; Make Entry and Table classes non-virtual
- Looks like they're not inherited, so we can save some memory by making them non-virtual.
- Entry getters are now constexpr
- Table is trivially constructive
- Merged ResetEntrySet into the destructor
- Put entries_ outside of a unique_ptr. EntryTrie uses its own hash map.

PiperOrigin-RevId: 513528653
2023-03-03 00:11:11 +09:00
Emma Haruka Iwao
70ab7ac046 Add HwndToUint32 function to cast HWND to uint32_t in 64-bit environments
On 64-bit Windows, HWND is 64-bit long, but only 32 bits are used. Mozc's IPC messages use 32-bit window handles. reinterpret_cast between objects with different sizes is not allowed in C++.

PiperOrigin-RevId: 513383634
2023-03-02 10:14:31 +09:00
Emma Haruka Iwao
0dc1550e76 Move SpecialKeyMap and related functions to special_key.cc
As we discussed in cl/513072642, we're moving special key related functions to special_key.cc.

- Add a new class, SpecialKeyMap
- Implementation of Table::ParseSpecialKey is now in SpecialKeyMap::Parse
- Move FindBlock, ParseBlock, and DeleteSpecialKey to special_key.cc
- Renamed DeleteSpecialKey as DeleteSpecialKeys
- Use std::optional and return by value
- Instead of using a cursor, update the input string with a substring

PiperOrigin-RevId: 513378557
2023-03-02 09:50:23 +09:00
Emma Haruka Iwao
dc71b36934 Use GetSourceFileOrDie to load font files. Change the paths to the same place as what Bazel data attribute uses.
PiperOrigin-RevId: 513372710
2023-03-02 09:24:57 +09:00
Emma Haruka Iwao
83ce486575 Add copts_wtl and features_gdi macros
These macro return copts and features values when building for Windows.
This CL also moves the mozc_win32_lib macro to the separate internal.bzl as it's not intended to be used outside of the directory.

PiperOrigin-RevId: 513368430
2023-03-02 09:07:35 +09:00
Hiroyuki Komatsu
961949a156 Update BUILD_OSS to 5029.
PiperOrigin-RevId: 513196141
2023-03-01 21:09:54 +09:00
Emma Haruka Iwao
8a4625c11f win32 renderer: Fix compile errors with clang
- Add a default statement to non-exhaustive switch statements
- Reorder constructor initialization list
- Include what you use

PiperOrigin-RevId: 513130389
2023-03-01 14:37:54 +09:00
Emma Haruka Iwao
45dd0436cc Fix a bug where the size of log_font passed to ZeroMemory was the size of the pointer
Changed the size to sizeof(LOGFONTW). It previously passed sizeof(log_font), but it's a pointer. Use the underlying type instead.

PiperOrigin-RevId: 513126095
2023-03-01 14:13:13 +09:00
Emma Haruka Iwao
eb33c21104 Add win32 library definitions for gdiplus, d2d1, dwrite.
Will be used by the Win32 renderer.

PiperOrigin-RevId: 513095185
2023-03-01 11:19:35 +09:00
Emma Haruka Iwao
90fb8f860d winmain: Force the Windows subsystem
The source file has a pragma comment for msvc, but it doesn't work for other compilers. All targets that depend on this winmain library will have the linkopt.

PiperOrigin-RevId: 513093371
2023-03-01 11:09:45 +09:00
Emma Haruka Iwao
325a0d300b Add a new Skylark stub for internal use
PiperOrigin-RevId: 513093006
2023-03-01 11:07:53 +09:00
Toshiyuki Hanaoka
3478c1cf3b Change the description of "#" and "♯ in mobile
PiperOrigin-RevId: 513092984
2023-03-01 11:07:45 +09:00
Emma Haruka Iwao
d201460733 Use ABSL_HAVE_ATTRIBUTE to pack PBGR32Bitmap where available
MSVC doesn't support the packed attribute, so we need to continue to include pshpack2.h.

PiperOrigin-RevId: 513092952
2023-03-01 11:07:34 +09:00
Emma Haruka Iwao
c58cfc7ea9 Explicitly cast LONG offset values to int32_t.
In 64-bit clang, Json::Value has both 32-bit and 64-bit constructors and lead to ambiguous conversion errors. Explicitly cast the LONG values to avoid the errors.

PiperOrigin-RevId: 513092810
2023-03-01 11:06:47 +09:00
Emma Haruka Iwao
562d07e55a Suppress msan for recursive
This is a false positive. The suspected variable is returned from the system library.

PiperOrigin-RevId: 513092782
2023-03-01 11:06:39 +09:00
Emma Haruka Iwao
480e3c6ff9 Remove incomatible spellchecker libraries from win32 builds
PiperOrigin-RevId: 513082007
2023-03-01 10:15:18 +09:00
Emma Haruka Iwao
801fc12846 Refactor CharChunk::AddInputInternal with string_view
CharChunk::AddInputInternal modified the input string with string::erase, but the time complexity of string::erase is O(n) (n is the remaining string length). The new implementation instead returns a pair<bool, string_view>, which has the loop indicator and remainder of the input. string_view::substr has constant complexity.

AddInput can also use the same rewrite, but it's for another CL as it's a public method.

Also moved Table::TrimLeadingSpecialKey to internal/special_key.{h,cc} as it's only used by table_test and char_chunk.

PiperOrigin-RevId: 513072642
2023-03-01 09:34:08 +09:00
Emma Haruka Iwao
57b30dbb50 Add strings::FrontChar
FrontChar takea a string_view and returns a pair of string_views
- first: the first character in the input
- second: the rest of the input

Util::SplitFirstChar32 will be refactored to use this method later.
PiperOrigin-RevId: 513054214
2023-03-01 08:21:07 +09:00
Emma Haruka Iwao
34e7e4f6db Refactor ExistenceFilter and BlockBitmap
- Move BlockBitmap to the internal namespace and include the declaration in the header file. The declaration is now in the header file to make ExistenceFilter trivially destructible.
- ExistenceFilter is now trivially destructible.
- ExistenceFilter::Size, BlockBitmap::Get, BlockBitmap::Set, and BitsToWords are now constexpr

PiperOrigin-RevId: 513053962
2023-03-01 08:20:02 +09:00
Tomoki Nakagawa
d03953ea50 Migrate //base:cpu_stats_main to mozc::Thread2.
Also includes misc refactorings e.g. `absl::Duration` flag and proper cancellation of threads.

PiperOrigin-RevId: 512898169
2023-02-28 21:52:02 +09:00
Tomoki Nakagawa
2e5ffcd5bc Migrate //dictionary:suppression_dictionary_test to mozc::Thread2.
Also:
- Migrate `std::to_string` to `absl::StrCat`.
- Removed `IsRunning` part of assertion since it really doesn't add any value
PiperOrigin-RevId: 512894919
2023-02-28 21:31:38 +09:00
Tomoki Nakagawa
511d2031e1 Migrate //config:config_handler_test to mozc::Thread2.
Also contains misc cleanups e.g. IWYU, constexpr and `T[]` -> `std::array<T, _>`.

PiperOrigin-RevId: 512849596
2023-02-28 17:19:54 +09:00
Emma Haruka Iwao
d6fb264b6a Fix include path for ScopedCOMInitializer
PiperOrigin-RevId: 512840209
2023-02-28 16:18:08 +09:00
Emma Haruka Iwao
05ed98af6d Reorder include order for Windows
windows.h needs to be before sddl.h, otherwise build fails with clang.

PiperOrigin-RevId: 512811718
2023-02-28 13:26:29 +09:00
Emma Haruka Iwao
a1d13e264d Fix a bug where temporary string object was assigned to string_view
PiperOrigin-RevId: 512811570
2023-02-28 13:25:49 +09:00
Emma Haruka Iwao
12293ed765 Remove deprecated and unused GetFrequency and GetTicks, make Stopwatch trivially constructible.
New code should use GetAbslTime(), which has nano-second resolution.

#codehealth

PiperOrigin-RevId: 512804846
2023-02-28 13:02:59 +09:00
Emma Haruka Iwao
55a716af38 Refactor; Use = default for eligible constructors and destructors
Make Louds::Node getters constexpr

PiperOrigin-RevId: 512795695
2023-02-28 12:10:38 +09:00
Emma Haruka Iwao
4fe04486f1 prediction: const std::string& -> absl::string_view
#codehealth

PiperOrigin-RevId: 512793164
2023-02-28 11:53:54 +09:00
Emma Haruka Iwao
8ac1e59168 Refactor IndicatorVisibilityTracker
- Merge InternalState into the main class as it's small and no need to new it.
- Use Stopwatch::GetElapsed instead of deprecated GetElapsedMilliseconds
- Make IsVisible constexpr.
- IndicatorVisibilityTracker is now trivially constructible and destructible.

#codehealth

PiperOrigin-RevId: 512789396
2023-02-28 11:29:58 +09:00
Emma Haruka Iwao
c4298f4439 Fix include path for ScopedCOMInitializer.
PiperOrigin-RevId: 512785533
2023-02-28 11:10:21 +09:00
Emma Haruka Iwao
767fe1d7cc composer: const std::string& -> absl::string_view
#codehealth

PiperOrigin-RevId: 512782202
2023-02-28 10:55:26 +09:00
Emma Haruka Iwao
2e39386094 Use win32::Utf8ToWide and WideToUtf8 instead of Util
#codehealth

PiperOrigin-RevId: 512732331
2023-02-28 07:28:42 +09:00
Emma Haruka Iwao
dfae1dc277 Change the test path from notepad.exe to kernel32.dll
notepad.exe doesn't exist on headless VMs. kernel32.dll is always present.
Add a matcher for better diagnostic messages

PiperOrigin-RevId: 512657557
2023-02-28 03:02:04 +09:00
Emma Haruka Iwao
42155015c2 Use string_view to compare TOKEN_SOURCE.SourceName.
string_view supports comparisons including null-termination characters. Use it instead of AreEqualArray.

#codehealth

PiperOrigin-RevId: 512652373
2023-02-28 02:42:19 +09:00
Hiroyuki Komatsu
4b66832edc Fix build failures of testing.
* follow-up to cl/512471568
* Rollback the change to the rule of "googletest".
* Use mozctest_test.cc as a test path.

#codehealth

PiperOrigin-RevId: 512636268
2023-02-28 01:38:23 +09:00
Tomoki Nakagawa
5d37b2af39 Migrate //base:singleton_test to mozc::Thread2.
Also fixed potential data race there.

PiperOrigin-RevId: 512617897
2023-02-28 00:09:51 +09:00
Emma Haruka Iwao
0282a2d822 Add a char version of OneCharLen to strings/unicode.h
This is used by many so it'g going to be the first function to have this new place. There were two OneCharLen definitions in util.cc and japanese_util.cc, so both of them were replaced by the new definition.

#codehealth

PiperOrigin-RevId: 512553462
2023-02-27 17:59:26 +09:00
Emma Haruka Iwao
a73f7c7c68 Add file::DeleteRecursively for tests and tools.
This new library recursively deletes all files under a directory. It supports Windows and Linux/MacOS with IShellItem and fts(3) respectively. It doesn't support iOS and is intended for tools and tests use.

PiperOrigin-RevId: 512553124
2023-02-27 17:57:06 +09:00
Toshiyuki Hanaoka
74b307737c Support '#' symbol of the data/symbol/ordering_rule.txt
PiperOrigin-RevId: 512539613
2023-02-27 16:30:31 +09:00
Emma Haruka Iwao
7ef96e6675 Move ScopedCOMInitializer to a separate header-only library.
It's used by many tools that don't need the rest of WinUtil, so it's better to be a separate file.

#codehealth

PiperOrigin-RevId: 512535059
2023-02-27 15:53:05 +09:00
Emma Haruka Iwao
ade2d8a1ef Add win32/wide_char for Win32 wide character functions.
These are reimplementations of the same functions in Util, but faster.

- Take std::wstring_view instead of const std::wstring& (absl doesn't have wstring_view).
- No extra memory copy. The old implementation used a temporary buffer before returning as std::string/wstring.
- Return by value. Most of the existing callers don't reuse buffers anyways.

PiperOrigin-RevId: 512519791
2023-02-27 13:49:47 +09:00
Emma Haruka Iwao
ff52a72053 Rename ScopedHandle::take to release and make the class movable.
release is the name used by other smart pointers, so we're aligning with them.
ScopedHandle is now movable so you can use std::move like std::unique_ptr.

#codehealth

PiperOrigin-RevId: 512507269
2023-02-27 12:56:40 +09:00
Emma Haruka Iwao
04ec4ae727 Add BUILD rules for Windows for internal builds.
PiperOrigin-RevId: 512502953
2023-02-27 12:29:44 +09:00
Emma Haruka Iwao
96f8b590f6 Remove extra ifdef/endifs because undef is a no-op when the macro is not defined.
#codehealth

PiperOrigin-RevId: 512491532
2023-02-27 10:48:50 +09:00
Emma Haruka Iwao
6effec3a3c Rewrite googletest with TempDirectory, support win32.
- Use TempDirectory to get a path to a temp directory.
- access(2) checks privileges with real user, not effective user. So, instead of checking it every time, added a unit test for MozcTestTest.
- googletest now sets both test_srcdir and test_tmpdir correctly on Windows.

PiperOrigin-RevId: 512471568
2023-02-27 07:24:21 +09:00
Emma Haruka Iwao
e50b8be585 Add TempDirectory and TempFile.
TempDirectory::Default tries several common places and returns the first available directory as a TempDirectory instance. It can be used to create unique temp directories and files. These created objects will be deleted when they go out of scope.

TempDirectory::Default tries TEST_TMPDIR first, then the following places:
- Win32: GetTempPath2 or GetTempPath
- POSIX: TMPDIR, /tmp, (/sdcard for Android), current directory

This is the first library in base/file. file_util will be split and reorganized into this directory.

PiperOrigin-RevId: 512467701
2023-02-27 06:42:41 +09:00
Emma Haruka Iwao
9a1f010cb7 Change NodeSwitchingHandler::GetModeSwitchingRule to return value
Define and use a struct type to return both display_mode and input_mode at once. The default value when a rule is not found is NO_CHANGE, so we can simplify the conditionals.

#codehealth

PiperOrigin-RevId: 512467262
2023-02-27 06:37:56 +09:00