[iOS] Added Support for XCResult #686

abhishekbedi1432 · 2022-09-16T21:02:14Z

Motivation

Marathon for iOS produces an amazing HTML report (in the outputDir) but sometimes we need the final xcresult bundle for many reasons (say integrating with the CI, uploading the result in repository, etc).

The xcresult bundles are stored at transient paths under Xcode's derived-data folder.

In this PR, we have added an enhancement of providing xcResultBundlePath parameter in the Marathon file (under iOS Vendor Config)

As a positive side effect, this helps to avoid flooding Xcode's derived data directory.

…est plan)

…tag-xcxresult-testplan-from-anton * ios-uitest-runner-via-tags: Update README.md Added Support to run iOS Test Runners Tag # Conflicts: # configuration/src/main/kotlin/com/malinskiy/marathon/config/vendor/VendorConfiguration.kt # sample/ios-app/Marathonfile

… into develop/talabat/final-tag-xcxresult-testplan-from-anton * develop/ios-support-for-test-plan-from-xcresult-branch: Modified XCTestRun to adapt to new format of xctestrun ( supporting test plan)

Malinskiy

Logic wise we're missing pulling back the xcresult back to the marathon's side

Malinskiy · 2022-10-19T09:15:53Z

README.md

 - **performance** using high parallelization (handling dozens of devices)

 ## Documentation

 Please check the official [documentation](https://marathonlabs.github.io/marathon/) for installation, configuration and more

+## [iOS Only] Added Support to consume tags for test functions under XCTestCase Subclasses


Please move the docs info to the https://github.com/MarathonLabs/marathon/blob/develop/docs/_posts/2018-11-19-ios.md. They're automatically published at https://marathonlabs.github.io/marathon/

Malinskiy · 2022-10-19T09:17:26Z

...guration/src/main/kotlin/com/malinskiy/marathon/config/serialization/ConfigurationFactory.kt

-                        ?: fileListProvider
+                    val resolvedResultBundlePath = marathonfileDir.resolve(configuration.vendorConfiguration.xcResultBundlePath)
+
+                    // Adding support for Test Plan


Please remove the comment. It doesn't make sense after we merge this.

Malinskiy · 2022-10-19T09:28:43Z

...guration/src/main/kotlin/com/malinskiy/marathon/config/serialization/ConfigurationFactory.kt

                            .fileList(resolvedDerivedDataDir)
-                            .firstOrNull { it.extension == "xctestrun" }
-                        ?: throw ConfigurationException("Unable to find an xctestrun file in derived data folder")
+                            .firstOrNull { it.extension == "xctestrun" && it.name.contains("$testPlanName") } ?: throw ConfigurationException("Unable to find matching TestPlan. Please recheck if testplan is enabled")


You can skip creating an additional string here:
it.name.contains("$testPlanName") -> it.name.contains(testPlanName)
It might also be a good idea to use startsWith or equals instead of contains. I'm not sure if there is a contract on the file name here

Malinskiy · 2022-10-19T09:31:59Z

core/src/main/kotlin/com/malinskiy/marathon/Marathon.kt

@@ -79,6 +80,10 @@ class Marathon(
        val tests = applyTestFilters(parsedTests)
        val shard = prepareTestShard(tests, analytics)

+        log.info("\n\n\n **** Marathon File Params **** \n")


This is a no-no: configuration might contain sensitive credentials, and we can't print them to the stdout

If you really want to do this then we need to implement some kind of masking for sensitive fields

Malinskiy · 2022-10-19T09:34:16Z

core/src/main/kotlin/com/malinskiy/marathon/test/Test.kt

@@ -6,7 +6,8 @@ data class Test(
    val pkg: String,
    val clazz: String,
    val method: String,
-    val metaProperties: Collection<MetaProperty>
+    val metaProperties: Collection<MetaProperty>,
+    val tags: Collection<String> = emptyList()


MetaProprety is an abstraction over tags, annotations and so on. We should just reuse MetaProperty here

Refer:
[iOS]Added Support for Cucumberish Tags / Function Annotations #702

Malinskiy · 2022-10-19T09:35:00Z

core/src/main/kotlin/com/malinskiy/marathon/Marathon.kt

-        return tests
+        val iosConfig = configuration.vendorConfiguration as? VendorConfiguration.IOSConfiguration
+
+        return tests.filter { it.tags.contains(iosConfig?.xcTestRunnerTag) }


This should just be an annotation filter, no vendor-specific logic in the core

Malinskiy · 2022-10-19T09:36:52Z

sample/ios-app/sample-appUITests/MoreTests.swift

@@ -27,6 +27,7 @@ class MoreTests: XCTestCase {
        super.tearDown()
    }

+    // @Flowers


Instead of modifying existing tests we should just add more tests with these tags

Malinskiy · 2022-10-19T09:38:38Z

vendor/vendor-ios/src/main/kotlin/com/malinskiy/marathon/ios/IOSDevice.kt

@@ -382,3 +390,11 @@ private fun String.toInetAddressOrNull(): InetAddress? {

 private fun TestBatch.toXcodebuildArguments(): String =
    tests.joinToString(separator = " ") { "-only-testing:\"${it.pkg}/${it.clazz}/${it.method}\"" }
+
+private fun getTimezone(): String {


This method sounds like getTimestamp rather than getTimezone

Malinskiy · 2022-10-19T09:41:08Z

vendor/vendor-ios/src/main/kotlin/com/malinskiy/marathon/ios/IOSTestParser.kt

@@ -20,6 +21,11 @@ class IOSTestParser(private val vendorConfiguration: VendorConfiguration.IOSConf
     *  marked as skipped in `xctestrun` file.
     */
    override suspend fun extract(): List<Test> {
+
+        if (vendorConfiguration.xcTestRunnerTag.isNullOrEmpty()) {
+            logger.warn { "[iOS] Did not find `xcTestRunnerTag` in iOS Vendor Config. This will result in running all tests" }


The tag filtering doesn't have anything to do with the responsibility of the parser, this probably doesn't belong here

Malinskiy · 2022-10-19T09:42:26Z

vendor/vendor-ios/src/main/kotlin/com/malinskiy/marathon/ios/xctestrun/Xctestrun.kt

+*/
+
+    private val targetMap: Map<String, *> = (
+        ((((((propertyList.valueForKeypath("TestConfigurations") as? Array<Any>)?.first()) as Map<*, *>)


first() will throw NullPointerException here when TestConfigurations is missing. Reproduced on the old sample

Malinskiy · 2022-10-19T09:47:51Z

Closes #620

…latform-Test-Runner-Marathon into develop/talabat/final-tag-xcxresult-testplan-from-anton

…final-tag-xcxresult-testplan-from-anton

abhishekbedi1432 · 2022-10-27T22:24:16Z

Pls refer: #700 for the feature (Adding support for XCResult). I will close this PR addressing all the comments in #700

abhishekbedi1432 added 3 commits August 23, 2022 19:14

Added Support to run iOS Test Runners Tag

553ca79

Update README.md

1f09d59

[iOS] Added Support for XCResult

4794292

abhishekbedi1432 requested review from Malinskiy, tagantroy and Vacxe as code owners September 16, 2022 21:02

abhishekbedi1432 and others added 8 commits September 17, 2022 01:02

Improved timestamp format

c0c66c3

Merge branch 'develop' into develop

7d64884

Modified XCTestRun to adapt to new format of xctestrun ( supporting t…

e5f73c9

…est plan)

Merge branch 'develop/ios-support-for-test-plan-from-xcresult-branch'…

fe05703

… into develop/talabat/final-tag-xcxresult-testplan-from-anton * develop/ios-support-for-test-plan-from-xcresult-branch: Modified XCTestRun to adapt to new format of xctestrun ( supporting test plan)

Added Test Plan to Sample App

dcbb081

Added XCTestPlan in iOS Vendor Config to support Test Plans

668ff5f

Merge branch 'develop' into develop

66d5239

Malinskiy requested changes Oct 19, 2022

View reviewed changes

Malinskiy and others added 4 commits October 20, 2022 13:14

Merge branch 'develop' into develop

ac5aeab

Fixed extracting testplan when Scheme & Target name are both same.

8488d9f

Merge branch 'develop' of https://github.com/abhishekbedi1432/Cross-P…

2e86c53

…latform-Test-Runner-Marathon into develop/talabat/final-tag-xcxresult-testplan-from-anton

Merge remote-tracking branch 'upstream/develop' into develop/talabat/…

8630afe

…final-tag-xcxresult-testplan-from-anton

abhishekbedi1432 closed this Oct 27, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[iOS] Added Support for XCResult #686

[iOS] Added Support for XCResult #686

abhishekbedi1432 commented Sep 16, 2022

Malinskiy left a comment

Malinskiy Oct 19, 2022

Malinskiy Oct 19, 2022

Malinskiy Oct 19, 2022

Malinskiy Oct 19, 2022

Malinskiy Oct 19, 2022

Malinskiy Oct 19, 2022

abhishekbedi1432 Oct 30, 2022

Malinskiy Oct 19, 2022

Malinskiy Oct 19, 2022

Malinskiy Oct 19, 2022

Malinskiy Oct 19, 2022

Malinskiy Oct 19, 2022

Malinskiy commented Oct 19, 2022

abhishekbedi1432 commented Oct 27, 2022

[iOS] Added Support for XCResult #686

[iOS] Added Support for XCResult #686

Conversation

abhishekbedi1432 commented Sep 16, 2022

Motivation

Malinskiy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Malinskiy commented Oct 19, 2022

abhishekbedi1432 commented Oct 27, 2022