Commit Graph

138 Commits

Author SHA1 Message Date
8293484a86 [*] Reworked AuAsync ThreadPool shutdown (again)
[*] Fixed nested lock under WorkItem
2023-02-08 18:23:37 +00:00
8b6c93fba9 [*] Fix: ThreadPool::ResolveHandle throws null 2023-02-06 04:07:54 +00:00
f43251c8fc [+] AuNet::ISocketChannelEventListener
[+] AuFS::UpdateTimes
[+] AuFS::UpdateFileTimes
[+] AuFS::CompressEx
[*] AuFS::Compress now rejects files that look to be already compressed
[+] AuFS::DecompressEx
[+] AuFS::Create
[+] AuFS::WriteNewFile
[+] AuFS::WriteNewString
[+] AuFs::FileAttrsList
[+] AuFs::FileAttrsGet
[+] AuFs::FileAttrsSet
[+] DirectoryLogger::uMaxLogsOrZeroBeforeCompress
[+] ISocketChannel.AddEventListener
[+] ISocketChannel.AddEventListener
[+] DirectoryLogger.uMaxLogsOrZeroBeforeCompress
[*] Fix UNIX regression
[*] Fix up stream socket channel realloc IPC
[*] Fix shutdown regression in pretty much everything thanks to 8ff81df1's dumbass fix
    (fixes fence regression on shutdown)
[*] Fix DirDeleterEx formatting of reported failed paths
[*] Fix up file not truncated if already exists bugs. Extended and alternative apis added.
[*] Fix ICompressionStream::ReadEx returning the wrong read value
[+] Legacy compression API can now self-correct once newer stream processor objects are added
2023-02-04 19:43:01 +00:00
8ff81df129 [*] Fix deadlock involving WaitFor under ThreadPool (shutdown race)
[*] Optimize mutex lock out of RWLockImpl::TryLockWrite
[*] Force all relevant members of RWLockImpl to be volatile just bc lol (afaik we cant justify it yet; however, i want to minimalize the risk of future issues in this type)
2023-01-30 14:35:48 +00:00
51a2816f3f [*] Merge GTK JS's branch fix for async shutdown
[+] IConditionMutex::TryLock()
... Timed is still too much of an ask for this interface, i think. There's a good reason as to why this isn't a iwaitable. This condvar interface must be as common and primitive as possible.
2023-01-30 13:26:17 +00:00
2474566d80 [*] two hotfixes 2023-01-29 23:03:54 +00:00
d4dfe22c6c [+] Always expand directory tree when attempting to copy or move files, just like every other FIO create operation.
[*] ~~Fix improper yield BlockUntilComplete on NewFence~~, kind of, not really. Wasn't really a bug. Might've introduced more overhead. Might revert.
2023-01-23 21:18:58 +00:00
f0fcec0cb7 [+] _CLOEXEC on F-tier operating systems predating SMP and processes 2022-12-16 03:48:04 +00:00
72dc0d715e [*] Begin enforcing steady time
[+] IOProcessor::WakeupThread
[+] NT: Begin hacking in timeBeginPeriod (must spam it in some places)
[+] ConsoleTTY (more specifically the win32 calls) are too slow to run on the mainthread. Delegate to worker.
[*] AuTime.CurrentClockSteady
[*] AuTime.CurrentClockSteadyMS
[*] AuTime.CurrentClockSteadyNS
2022-11-28 16:01:08 +00:00
bb9c383aee [+] ICompressionInterceptor::LimitPassthroughOnOverflow
[+] IOPipeRequest::uMinBytesToRead
[+] (secret api intended for unix users) AuIO::NewLSOSHandleEx
[*] Fix quirks when running Gtk under an io processor
...CtxYield shouldn't spin while work (improper breakout on remote update)
[*] EFileOpenMode::eWrite should assume O_CREAT semantics making eReadWrite somewhat redundant. OpenWrite + eWrite should reasonably work with file-appends. it should not mean force create + cucked GetLength().
2022-11-20 10:31:13 +00:00
f86665fd36 [+] Net: TCP servers can now be multi-threaded
[+] Net: Added missing UDP send datagram
[*] IO bug fixes
2022-11-17 20:58:48 +00:00
48a8e4ae5a [+] Catch around all async DispatchFrames. Will add optional forced panic later.
[*] Fix thread handle bug on non-au threads
2022-10-14 07:16:28 +01:00
050e938f0c [*] transition some of the old vector based AuByteBuffer APIs to be stream aware
[*] fix: runnersRunning was uninitialized
[*] fix: unimplemented path in GetOrAllocateLinearWriteable
2022-09-19 06:42:39 +01:00
30da4bce04 [*] Fixup schedule failure rejection. Fixes crash during safe/good clean up. 2022-08-13 22:59:44 +01:00
bfd9e339fc [*] Clean up of include headers 2022-08-02 05:58:00 +01:00
2470cbc12a LAAAAAIIIIIIIIN 2022-07-21 23:23:16 +01:00
bf6f13095c [*] Linux builds again. Still behind 2022-06-23 13:25:20 +01:00
0c3344fe46 [+] IIOSimpleEventListener
[+] IOProcessor::StartSimpleIOWatch(const AuSPtr<IIOWaitableItem> &object, const AuSPtr<IIOSimpleEventListener> &listener)
[+] IOProcessor::StartSimpleLSWatch(const AuSPtr<Loop::ILoopSource> &source, const AuSPtr<IIOSimpleEventListener> &listener)
[*] IOAdapterAsyncStream should reset the transactions IO state upon reaching end of segment (should this be per tick?) or upon stream error - otherwise, we end up spinning on a stuck event forever
[*] Fix non-linear path under read of the AuByteBuffer
[*] Fix various other nonlinear conditions under AuByteBuffer
[*] IOProcessor releases registered io item from queue upon request
[*] Fix ConsoleMessage::Write -> enumeration of color should be casted to a uint8
[+] Error telemetry under async task creation
[*] Fix various lock ups and non-blocking spins related to erroneous InternalRunOne impl. Residual preemptive batching was fucking with modern io.
[*] Cleanup TaskFrom/JobFrom. More work required to clean up legacy piss
2022-06-22 14:42:17 +01:00
7fde7d04fb [+] EPipeCallbackType
[+] IIOBufferedStreamAvailable callback
[+] IIOProcessor singleshot work items / IIOProcessorWorkUnit
[+] IOPipeCallback description of a pipes destination
[+] IOPipeInputData description of a pipes source
[+] IOPipeRequest, IOPipeRequestAIO, IOPipeRequestBasic
[+] IPipeBackend hooks for on start/end hooks of IOPipeRequestBasics
[*] Update IOAdapaterAsyncStream implementation to better support caller buffering
[*] Updated IAsyncStreamReader to include a warm/dequeue API for direct async usage
[*] Fix NT IO regressions
[*] Fix ThreadPool shutdown on an unregistered thread
[*] Fix race condition in Async.NT.cpp & fix signalable state to closely match Linux (dunno how this was passing before)
[*] Refactor IOProcessorWorkUnit -> IIOProcessorWorkUnit
[*] Update experimental header to include the changes
2022-06-21 05:49:36 +01:00
8bd7f698e3 [*] Merge branches (includes fixes) 2022-06-12 19:23:18 +01:00
5d2b642ac0 [*] Huge refactor - Move Aurora::[IPC and Loop] to Aurora::IO 2022-06-12 01:29:28 +01:00
44108a322e 2/3 of the IO update (very early implementation)
[+] TTYConsole::GetPaddingTopOfLog,GetPaddingHeadOfLog,GetPaddingTopOfLog [+ set variants]
[+] IO::IOYield()
[+] IO::IAsyncTransaction::Failed,GetOSErrorCode()
[+] IByteBufferStreamPair
[+] IIOBufferedInterceptor
[+] IIOBufferedProcessor
[+] IIOEventListener
[+] IIOPipeEventListener
[+] IIOProcessorEventListener
[+] IIOProcessorManualInvoker
[+] IIOWaitableIOLoopSource
[+] IIOWaitableIOTimer
[+] IIOWaitableItem
[+] IIOWaitableTickLimiter
[+] IOAdapterAsyncStream
[+] IOAdapterByteBuffer
[+] IOAdapterCompression
[+] IOAdapterSeeking
[*] Cleanup CpuInfo.Linux.cpp
[*] Fixup async threadpool some more
[*] LSTimer.NT.cpp updates timer object on tick state update, akin to Linux
2022-06-12 00:01:27 +01:00
342db487f9 [*] AuAsync invalid self itr upon nested poll 2022-05-20 01:31:58 +01:00
4079d471a7 [*] Fix a deadlock in auasync 2022-05-19 04:07:10 +01:00
59ea8d1c3d [+] Try/catch around SysPump under async sched 2022-05-18 21:22:58 +01:00
de504a3a2d [*] AuAsync: remove illegal static casts 2022-05-17 02:43:40 +01:00
00c82fdbe4 [*] Update WorkItem ::WaitFor 2022-05-17 01:59:19 +01:00
2c1d22ac13 [*] Missing AU_LOCK_GUARD in IWorkItem::WaitFor non-array variant 2022-05-17 00:46:25 +01:00
286ae14a7b [*] Refactor WorkItemHandler EProcessNext -> ETickType
[*] AuAsync aue 1 regression
2022-05-17 00:41:27 +01:00
b98baea2a8 [*] 4/6 fatal bugs in auAsync 1.0 grime 2022-05-10 14:51:22 +01:00
eeaa10090d [*] 2/4 critical bugs in legacy aurora engine async subsystem 2022-05-10 14:22:37 +01:00
1c7267b9d8 [*] Update how AuAsync handles SysPump - we should really deprecate it. 2022-05-09 19:27:36 +01:00
3defb1bb14 [+] Linux Watcher
[*] Expand watcher API -> Breaking NT
[*] Reexpand loop queue API -> Breaking NT
[*] Linux CPUInfo clean up
[*] Bug fix: mkdir should set execute flag... because directories are special
[*] Refactor: Cleanup base64
[*] Bug fix: UNIX path normalization
[*] Bug fix: missing O_CREAT flag (au auto-creates)
[*] Normalize line endings
2022-04-10 16:40:49 +01:00
e25d8a65c8 [*] Linux Progress... It compiles (APIs are out of date, subsystems are missing, and it doesn't link. Worse than being 5m behind) 2022-04-05 11:11:19 +01:00
5599da3ab0 [*] Harden exit 2022-04-02 01:48:29 +01:00
3507d9bb33 [+] GetSharedAsyncApp 2022-03-23 16:33:02 +00:00
03bb80239c [*] Reglue async and loop together 2022-03-10 15:41:57 +00:00
7864966478 [+] Minimalistic AuNumericLimits implementation 2022-02-19 17:56:45 +00:00
b29be7b2d5 [*] Hardened line bufferer
[+] Unstaged changes from last commit
2022-02-19 11:50:14 +00:00
be7e9271e6 [+] Added casts for pointer types
[*] Nuke more std:: references
2022-02-19 11:43:57 +00:00
cf70f0d45c [*/+/-] MEGA COMMIT. ~2 weeks compressed.
The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api.

[+] AuMakeSharedArray
[+] Technet ArgvQuote
[+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.)
[+] auEndianness -> Endian swap utils
[+] AuGet<N>(...)
[*] AUE_DEFINE conversion for
        ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump
[+] ConsoleMessage ByteBuffer serialization
[+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks
[*] Split logger from console subsystem
[+] StartupParameters -> A part of a clean up effort under Process
[*] Refactor SysErrors header + get caller hack
[+] Atomic APIs
[+] popcnt
[+] Ring Buffer sink
[+] Added more standard errors
        Catch,
        Submission,
        LockError,
        NoAccess,
        ResourceMissing,
        ResourceLocked,
        MalformedData,
        InSandboxContext,
        ParseError

[+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace
[+] IExitSubscriber, ETriggerLevel
[*] Write bias the high performance RWLockImpl read-lock operation operation
[+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem)
[*] Updated API style
        Digests
[+] CpuId::CpuBitCount
[+] GetUserProgramsFolder
[+] GetPackagePath
[*] Split IStreamReader with an inl file
[*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer
[+] ICharacterProvider
[+] ICharacterProviderEx
[+] IBufferedCharacterConsumer
[+] ProviderFromSharedString
[+] ProviderFromString
[+] BufferConsumerFromProvider
[*] Parse Subsystem uses character io bufferer
[*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore]
[+] ByteBuffer::ResetReadPointer
[*] Bug fix bytebuffer base not reset on free and some scaling issues
[+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section
[*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing
[+] Added 64 *byte* fast RNG seeds
[+] File Advisorys/File Lock Awareness
[+] Added extended IAuroraThread from OS identifier caches for debug purposes
[*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions.
[*] Broke AuroraUtils/Typedefs out into a separate library
[*] Update build script
[+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media
[*] Improved public API documentation
[*] Bug fix `SetConsoleCtrlHandler`
[+] Locale TimeDateToFileNameISO8601
[+] Console config stdOutShortTime
[*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl)
[*] Bug fixes in decoders
[*] Major bug fix, AuMax
[+] RateLimiter
[+] Binary file sink
[+] Log directory sink
[*] Data header usability (more operators)
[+] AuRemoveRange
[+] AuRemove
[+] AuTryRemove
[+] AuTryRemoveRange
[+] auCastUtils
[+] Finish NewLSWin32Source
[+] AuTryFindByTupleN, AuTryRemoveByTupleN
[+] Separated AuRead/Write types, now in auTypeUtils
[+] Added GetPosition/SetPosition to FileWriter
[*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp
[*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position)
[*] Hack back in the sched deinit
[+] File AIO loop source interop
[+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw
[+] Stub code for networking
[+] Compression BaseStream/IngestableStreamBase
[*] Major: read/write locks now support write-entrant read routines.
[*] Compression subsystem now uses the MemoryView concept
[*] Rewrite the base stream compressions, made them less broken
[*] Update hashing api
[*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing
[+] Added new AuByteBuffer apis
    Trim, Pad, WriteFrom, WriteString, [TODO: ReadString]
[+] Added ByteBufferPushReadState
[+] Added ByteBufferPushWriteState
[*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16.
[*] ELogLevel is now an Aurora enum
[+] Raised arbitrary limit in header to 255, the max filter buffer
[+] Explicit GZip support
[+] Explicit Zip support
[+] Added [some] compressors

et al
2022-02-17 00:11:40 +00:00
7eb6900e9f [+] Added shared pointer extension, begin experimenting defining throw-on-null mechanic
[+] AuRemoveConst
[*] Support circular reference in Aurora pipelines Include, added support for early Aurora::Build AuroraEnum
[+] Added SWInfo API
[+] AU_COPY_MOVE, AU_MOVE, AU_COPY to go with AU_NO_... variants
[+] Adding GetProcessId
2022-01-26 00:22:02 +00:00
e5e36bd887 Large Commit
[*] Fix deadlock in the async subsystem (NoLockShutdown vs Shutdown in exception handler)
[+] Added ProccessMap NT variant
[+] Added ToolHelp image profiling
[*] Improved exception awareness
[*] Delegated SpawnThread to isolated TU, ready for reuse for RunAs and XNU Open - now with horrible evil alloc that could fail
[+] Added header for future api 'UtilRun'
[*] Improve NT core detection
[*] Changed small affinity bitmap to AuUInt64 instead of AuUInt32
[+] Added data structure to hold cpuids/affinity masks
[+] Implemented logger sinks
[+] Implemented logger glue logic
[*] Began migrating older loggers to sink-based default devices
[*] Minor refactors
[*] Improved internal exception discarding, not yet nothrow capable
[*] Minor create directory fix
2022-01-24 18:43:53 +00:00
c965d8384f [+] AuStaticPointerCast
[*] More refactoring
2022-01-19 18:53:22 +00:00
6b9bddeeaf nuke more of the stl 2022-01-19 18:18:13 +00:00
7759d38ed0 [*] Exterminate more std:: references 2022-01-19 17:08:13 +00:00
f1040a67b0 ...cont 2022-01-19 11:47:29 +00:00
6135618e24 [+] Added AuIsThreadRunning
[*] Fix path bug
[*] Fixed two async shutdown bug
2022-01-19 02:49:44 +00:00
77775410ec [+] Added some notes about safer low memory condition handling
[*] Adjusted code style in MSDN reference code
2021-12-26 15:51:31 +00:00
d4ec3749e7 [*] fix new build chain build issues
[-] test trash from workpairimpl
[+] Process.hpp impl stub
2021-11-15 13:07:38 +00:00
4282bb3fa6 [+] Added reschedClockAbsMs/reschedClockAbsNs
[*] Added AU_TEMPLATE_ENABLE_WHEN to utility APIs where there are PId and Id overlods
2021-11-08 00:11:01 +00:00
22103a8ad3 [*] This this this 2021-11-07 20:26:54 +00:00
320d6b95ce [-] Remove experiment that was accidently left in Entrypoint.cpp 2021-11-06 02:20:19 +00:00
99c5e1fa65 A pretty large patch not worth breaking up into separate commits
[*] Split up Aurora Async
[*] Split Async app into seperate ThreadPool concept
[*] Fix various OSThread bugs and tls transfer issues
[*] Set default affinity to 0xFFFFFFFF
[*] Update Build script
[+] Add AuTuplePopFront
[+] New Network Interface (unimplemented)
[*] Stub out the interfaces required for a better logger
[*] Fix Win32 ShellExecute bug; windows 11 struggles without explicit com init per the docs - now deferring to thread pool
[*] Update gitignore
[*] Follow XDG home standard
[*] Refactor some namespaces to use the shorthand aliases
[*] Various stability fixes
2021-11-05 17:34:23 +00:00
fbe151506f [+] Missed unrefactered chunk of code; locale build cache wasn't invalidated for some reason 2021-10-24 11:44:43 +01:00
b8fde0bdf0 [*] Refactor and clean up exception and threading API
[-] Deprecated abstract vectors struct
2021-10-24 11:19:47 +01:00
5e0cc1ccfa [-] Comment out noted problematic code. A fix must be implemented soon
[*] Harden Async.hpp utilities against various conditions one might expect in the wild
[+] Added SetWorkerIdIsThreadRunner
[*] Fix a bug where a cv signal and broadcast was inverted
2021-10-16 00:02:27 +01:00
f559897b42 [+] Added WorkerId_t structure
[-] Removed WorkerId_t typedef
[*] Added shared support to some older threading apis that have yet to be updated
2021-10-08 20:51:34 +01:00
f968d6a75f Merge remote-tracking branch 'origin/master' 2021-10-06 14:48:38 +01:00
c9e1dc30e0 [+] Added AU_STRIP and AU_EMIT macros
[+] Added AuroraForEach
[+] Added AuroraInterfaces
[+] Added Async::NewFence
[*] Readded task null check under Async::NewWorkItem such that task = AuMakeShared() errors are passed through to the caller by a failure empty shared response
2021-10-06 14:48:28 +01:00
499f553e15 [*] Refactor AddLoopSource 2021-10-03 13:43:58 +01:00
15ae6ab14c [+] Added file structure for the loop source subsystem 2021-10-02 11:28:49 +01:00
a6d1c04ba0 [+] Added AuExists semantic 2021-10-01 17:49:23 +01:00
ea83fcef20 [...] cont 2021-10-01 17:32:50 +01:00
9436b12455 [+] Begin work on a possible future API for interoperability between loop sources and event driven async apps 2021-10-01 17:27:15 +01:00
caa983ecad [*] Major change in include convention 2021-09-30 15:57:41 +01:00
6e5905b80a [*] Permit unbacked IWorkItems 2021-09-30 11:29:18 +01:00
3fb8e2bf8c Remove ID 0 restriction from Aurora async, breaking the API 2021-09-30 10:19:04 +01:00
2c5a492f08 [*] Added a few notes in the Async.hpp header
[+] Ensure dead locks can not occur
2021-09-29 15:31:40 +01:00
d3a97257da God dammit 2021-09-29 14:43:49 +01:00
5347fdbba1 [+] Enforce C++ assumptions 2021-09-29 14:40:30 +01:00
dca4ca195b [+] Added async task (this should be job) private
[+] Added ToWorkResultT for apps using BasicWorkCtx
2021-09-29 11:47:54 +01:00
8a6abcfdb7 [+] Added IWorkItem::Cancel 2021-09-29 09:01:42 +01:00
ffe4bc31fc Major patch [1/2] 2021-09-06 11:58:08 +01:00
3e33533276 [-] Remove a scope I was using with SysBenchmark 2021-07-15 17:28:45 +01:00
72a74eb7a4 [*] Fixed bug where schedular was using a read lock in a scope where items were erased from a vector
[*] Refactor a 'Object' member field in a ParseObject struct
[+] Added an option to set a command dispatcher thread from an AsyncApp
[*] Fix various issues with AsyncApp, stablity improvements
[+] Added AddDelayTime
2021-07-15 17:16:23 +01:00
28a840f71d [+] Added async semantic ->Then(...) 2021-07-12 15:37:05 +01:00
7579d70bc1 [*] Actually compiles now :( 2021-07-11 18:26:38 +01:00
df301a4101 Improve workitem api
[+] New waitfor by work dispatcher type
2021-07-07 21:32:59 +01:00
1f8d06cbf0 [*] Expose AsyncApp::Poll 2021-07-05 14:35:13 +01:00
18c435a433 [+] Sketchy pump system io events in ::Main of async apps 2021-07-01 14:49:32 +01:00
9a93d4ec8d [+] Added the old RSA wrappers
[*] Prepare for implementing ECC (again)
2021-07-01 10:18:42 +01:00
c8c3908085 [+] Now with a scheduler! 2021-06-30 13:07:54 +01:00
e30be64c44 [*] Prevent pinning another WorkItems lock within the parent scope of some ::Fail's 2021-06-30 10:41:58 +01:00
23ddb0f2f6 [*] Ensure failure on waiters if a task we're waiting on has already failed 2021-06-30 10:34:01 +01:00
452b49b610 [+] Begin work on the Async subsystem 2021-06-30 10:28:52 +01:00
337062b490 Update headers, untabify. I guess the vs extension killed itself 2021-06-27 22:33:58 +01:00
eb4a495199 Initial Commit 2021-06-27 22:25:29 +01:00